Atomic compare‑exchange stalls when a timeout is enforced on AArch64
0 reputation · 19 Oct 2024, 23:55 UTC
0 reputation · 19 Oct 2024, 23:55 UTC
In a Linux userspace program compiled for ARMv8‑A, a thread repeatedly executes a C++ std::atomic::compare_exchange_weak inside a loop that also checks a deadline and a cancellation flag. When the operation is contended on a big core, the loop appears to spin forever, even though the deadline has elapsed and the cancellation flag is set.
The architecture offers two atomic models: the baseline LL/SC pair (LDXR/STXR) and the Large System Extensions (LSE) RMW instructions (CAS, LDADD). The compiler switches between them based on -march/-mcpu and the target feature macros. If the target is LSE‑capable, the generated code uses a single RMW instruction; otherwise it emits a retry loop that may never observe the cancellation flag if the loop is not properly interleaved with a memory barrier.
Because the cancellation logic is implemented in user space, it relies on the same memory ordering as the atomic operation. A mismatch can make the timeout check never see the updated flag, leading to a silent hang.
What are the documented conditions under which a LL/SC loop can block indefinitely in the presence of a deadline? How does LSE affect the visibility of a cancellation flag in a timeout loop? What compiler or runtime flags can guarantee that a timeout check will be observed even when the atomic operation uses a retry loop?
33275 reputation · 20 Oct 2024, 11:50 UTC
An LL/SC (LDXR/STXR) loop can spin forever when another core repeatedly wins the store‑exclusive, causing the loop to retry without observing unrelated stores such as a cancellation flag. This livelock is possible because the LL/SC model does not guarantee forward progress; only executing a WFE/WFI or a separate memory‑ordering instruction provides the architectural guarantee of eventual progress.
When the Large System Extensions (LSE) are available, the compiler emits a single CAS RMW instruction (CASP/CASL). This instruction completes in bounded steps and carries implicit acquire/release semantics, so a concurrently updated cancellation flag becomes visible before the CAS returns, preventing the indefinite stall.
To guarantee that a timeout check is observed even when the code must fall back to an LL/SC retry loop, either:
Confirmed facts (from the ARM Architecture Reference Manual and LSE specification):
Likely explanation for the observed hang:
The program’s timeout/cancellation flag is read with the same memory ordering as the atomic compare‑exchange. In a pure LL/SC loop without an intervening fence or yield, the core may repeatedly win the load‑exclusive but lose the store‑exclusive, causing the loop to retry and never re‑load the cancellation flag, so the deadline check appears stale.
while (!atomic.compare_exchange_weak(expected, desired)) { // read cancellation flag with acquire semantics if (cancel_flag.load(std::memory_order_acquire)) break; // optional yield to reduce contention std::this_thread::yield(); // explicit acquire fence before next LDXR std::atomic_thread_fence(std::memory_order_acquire); }This ensures that each iteration observes any update to the cancellation flag before attempting another LDXR/STXR pair.
Missing diagnostic detail
To confirm which path the binary is taking, we need to know: Is the binary built for an LSE‑capable core (‑march=armv8.2‑a or ‑march=armv8‑a+lse)? Knowing this determines whether the fix is simply enabling LSE or whether we must add the yield/fence loop.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.