ARM64's Weak Memory Model: Why x86 Code Fails on AArch64

The port that passes CI and fails in production

You move a multithreaded C++ service from x86-64 to an ARM64 server. Unit tests pass. Integration tests pass. Then, under production load, a consumer thread reads a stale value from a shared buffer. The producer wrote the payload, then set a ready flag. On x86-64 the hardware guaranteed that other cores would see those two stores in order. On AArch64 that guarantee does not exist.

The lesson is blunt: ARM64 is a weakly ordered memory platform. Correctness comes from explicit acquire/release or seq_cst atomics, never from hoping the hardware preserves program order.

x86-64 and AArch64 make different promises

x86-64 uses a Total Store Order (TSO) model. A store becomes visible to other cores in program order, and a load is not reordered ahead of an older store to a different address. This means a simple flag handshake often works by accident.

ARMv8-A defines a weak model. Plain loads and stores may be observed out of order by other cores. The processor can buffer a store and allow a younger store to become visible first, or a load can complete before an earlier store is globally visible. The architecture does not promise that "write A then write B" means every observer sees A before B.

What AArch64 actually gives you

The baseline ARMv8-A instruction set includes three relevant tools:

• Barriers: DMB (data memory barrier), DSB (data synchronization barrier), and ISB (instruction synchronization barrier).
• Ordered accesses: LDAR (load-acquire) and STLR (store-release).
• Exclusive pairs: LDXR/STXR and LDAXR/STLXR for load-linked/store-conditional loops.

ARMv8.1 adds Large System Extensions (LSE) atomics such as LDADD, SWP, and CAS. These perform an atomic read-modify-write in a single instruction, avoiding the retry loop that exclusive pairs require under contention.

You rarely write these instructions by hand. C11/C++11 atomics and Linux kernel primitives map to them:

Language / kernel construct	AArch64 result
memory_order_relaxed RMW	LSE instruction or LDXR/STXR loop
Acquire load / release store	LDAR / STLR
memory_order_seq_cst load/store	LDAR / STLR
atomic_thread_fence(seq_cst)	DMB ISH

Use std::atomic or kernel helpers like smp_load_acquire and smp_store_release. Do not use volatile: it creates no happens-before edge and does not stop the compiler from reordering plain accesses.

A message-passing slot that breaks

Consider a single-producer/single-consumer ring slot. The writer fills a payload, then publishes it with a plain flag. The reader spins on the flag, then reads the payload.

// Broken: plain shared variables
struct Slot {
    int payload;
    int ready;
};

void produce(Slot* s, int value) {
    s->payload = value;   // plain STR
    s->ready = 1;         // plain STR
}

void consume(Slot* s) {
    while (!s->ready) {}   // plain LDR
    use(s->payload);      // plain LDR
}

On x86-64 the hardware ordering hides the bug. On AArch64 the consumer can observe ready == 1 before the payload store is visible, then read a stale value.

The fix is to make the flag an atomic with release/acquire semantics:

#include <atomic>

struct Slot {
    int payload;
    std::atomic<int> ready{0};
};

void produce(Slot* s, int value) {
    s->payload = value;
    s->ready.store(1, std::memory_order_release); // STLR
}

void consume(Slot* s) {
    while (s->ready.load(std::memory_order_acquire) != 1) {} // LDAR
    use(s->payload);
}

The release store on the writer side and the acquire load on the reader side create a happens-before edge. Every write before the release is visible to the thread that performs the acquire.

Strong ordering is a throughput tax

It is tempting to make every shared variable seq_cst or sprinkle DMB ISH everywhere. That trades a rare heisenbug for a permanent performance penalty. DMB ISH and sequential-consistency operations serialize memory traffic and cost substantially more than relaxed or ordered accesses. The cost varies by microarchitecture and coherency fabric, so measure on your target part rather than trusting generic cycle counts.

The opposite failure is worse. One missing barrier produces bugs that appear only under load, on specific core counts, or on particular silicon. The goal is not maximum barrier strength; it is the correct minimum ordering for each data flow.

ARM64 is not one uniform target

Portable AArch64 code must assume only ARMv8.0 instructions. LSE atomics require ARMv8.1 or later. Cortex-A53 and Cortex-A72 class parts predate ARMv8.1, so an ARM64 binary is not automatically portable across the family.

Before relying on LSE, check the target. On Linux, read the CPU features:

grep -m1 Features /proc/cpuinfo | tr ' ' '\n' | grep atomics

Run this in a shell on the target aarch64 system. If the output contains atomics, the hardware supports ARMv8.1 LSE instructions. Alternatively, call getauxval(AT_HWCAP) and test HWCAP_ATOMICS at runtime. Compile with -march=armv8-a for baseline compatibility, or -march=armv8.1-a when you know the hardware.

How to verify the fix

Start by inspecting the generated assembly. Compile the example with:

gcc -O2 -S -march=armv8-a -o slot.s slot.cpp

Run this on an aarch64 system or a cross-compilation host. In the broken version, the flag accesses should appear as plain STR/LDR. In the fixed version, they should appear as STLR and LDAR (or a DMB ISH if you used a fence). If you compile the same source with -march=armv8.1-a, relaxed atomic read-modify-write operations should collapse from an exclusive loop into a single LSE instruction.

Next, stress the code on real hardware. Weak-memory bugs are probabilistic, so a single pass proves little. Pin producer and consumer to different cores and loop the handshake many times:

taskset -c 0,3 ./slot_test

Run this in a shell on the target system. The core IDs are placeholders; choose two distinct cores from your machine. The broken version may eventually fail the payload check. The fixed version should not.

Be skeptical of emulation. QEMU user-mode and many TCG system emulations do not faithfully model ARM's weak memory ordering, so passing tests under emulation does not prove correctness. Validate on real hardware or with a formal memory-model tool such as the herd tool from the diy7 suite.

Finally, remember that compiler reordering is separate from CPU reordering. Even correct hardware barriers do not stop the compiler from moving plain accesses. Ordering must be expressed through atomics or explicit compiler barriers.

Actionable takeaways

1. Audit every variable shared across threads. If it is not an atomic, it is a data race.
2. Replace plain flag handshakes with release-store / acquire-load pairs.
3. Inspect the disassembly to confirm STLR/LDAR (or DMB ISH) appear where you expect them.
4. Check /proc/cpuinfo or AT_HWCAP before using ARMv8.1 LSE atomics.
5. Stress test on real aarch64 silicon with core pinning; do not trust emulation or a single clean run.