Lock‑Free Circular Buffers in Embedded C: A Practical Guide for ISR‑to‑Main‑Loop Data Exchange

Why a Circular Buffer?

In most microcontroller projects the UART, SPI, or I²C peripheral generates data in an interrupt service routine (ISR) that the main loop later consumes. A circular buffer (ring buffer) lets the ISR push bytes without blocking and the main loop pull them later, keeping the system responsive. The key challenge is to keep the buffer operations lock‑free and safe on both 8‑bit and 32‑bit cores.

Problem & Thesis

When the ISR writes a byte while the main loop reads, the head and tail indices can race. The classic solution is to use a single‑producer/single‑consumer (SPSC) design that updates the indices atomically and avoids disabling interrupts for long periods. This blog explains how to size the buffer, choose the right index type, enforce memory ordering, and detect full/empty conditions without extra state.

1. Buffer Layout & Power‑of‑Two Size

A circular buffer is a fixed‑size array with two indices: head (write pointer) and tail (read pointer). To wrap indices efficiently, choose a size that is a power of two. Then idx & (size-1) replaces the expensive modulo operation.

/* Example for a 128‑byte UART RX buffer */
#define BUF_SIZE 128
#define BUF_MASK (BUF_SIZE - 1)

static volatile uint8_t rx_buf[BUF_SIZE] __attribute__((aligned(32)));
static volatile uint8_t head = 0;
static volatile uint8_t tail = 0;

Aligning the buffer to a 32‑byte boundary helps on dual‑core MCUs (e.g., STM32H7) by preventing false sharing of cache lines.

2. Head/Tail Atomicity

On 8‑bit MCUs a single byte index is naturally atomic. On 16‑bit or 32‑bit cores, a multi‑byte index may be split into two 8‑bit halves if the compiler does not guarantee atomicity. Two common strategies:

• Disable interrupts briefly while updating the index. For SPSC this is safe because only the ISR writes head and only the main loop writes tail.
• Use compiler built‑ins like __atomic_store_n or __sync_synchronize() to enforce a memory barrier around the index update.

Example disabling interrupts (CMSIS style):

void uart_isr(void) {
    uint8_t next = (head + 1) & BUF_MASK;
    if (next != tail) {            // buffer not full
        rx_buf[head] = (uint8_t)USART->DR;
        __DMB();                  // ensure store completes before index update
        head = next;
    } else {
        // optional: increment drop counter
    }
}

3. Detecting Full vs. Empty

A common convention is to leave one slot unused: the buffer is full when (head + 1) & BUF_MASK == tail. This eliminates the need for a separate count variable and keeps the head/tail updates single‑word.

Alternatively, keep a count variable, but that requires another atomic update and increases code size. For most ISR‑to‑main scenarios the sacrificed‑slot method is simpler and faster.

4. Memory Barriers & Weakly‑Ordered Cores

On Cortex‑M3/M4/M7 or RISC‑V cores the compiler or CPU may reorder memory accesses. The __DMB() (Data Memory Barrier) or GCC's __sync_synchronize() forces the write to the buffer to complete before the index update. Place the barrier immediately after writing the data, before updating head or tail.

5. Worked Example: UART RX ISR

Assume a UART peripheral that triggers an interrupt on each received byte. The ISR pushes the byte into rx_buf and the main loop processes complete messages.

/* UART ISR – called by the hardware vector table */
void USART1_IRQHandler(void) {
    uint8_t next = (head + 1) & BUF_MASK;
    if (next != tail) {
        rx_buf[head] = (uint8_t)USART1->DR;
        __DMB();
        head = next;
    } else {
        // buffer full – drop byte or increment overflow counter
    }
}

/* Main loop – called from the scheduler or while(1) */
void process_uart(void) {
    while (tail != head) {
        uint8_t byte = rx_buf[tail];
        __DMB();
        tail = (tail + 1) & BUF_MASK;
        handle_byte(byte);   // user logic
    }
}

Compile with -O2 -Wall -Wextra and inspect assembly to confirm that head and tail updates are single word, and that __DMB() is present.

6. Trade‑Offs & Limitations

• One slot wasted – capacity is BUF_SIZE - 1. For very high throughput you may need a larger buffer.
• ISR latency – the ISR does only one store and a branch. If you need to handle bursts at 115200 baud, keep the ISR minimal or switch to DMA ping‑pong.
• Atomicity on 8‑bit MCUs – if the compiler generates 16‑bit accesses for head or tail, disable interrupts around the update or use volatile uint16_t with a brief mask.
• Cache maintenance – on Cortex‑M7 dual‑core MCUs, DMA ping‑pong buffers must be cleaned/invalidated. Forgetting this causes stale reads.

7. Practical Verification Steps

1. Compile with -O2 -Wall -Wextra and use objdump -d to confirm head and tail updates are atomic.
2. Stress test – generate a high‑rate UART stream, run the firmware, and verify a checksum of all received bytes matches the transmitted data.
3. Measure ISR latency – toggle a GPIO in USART1_IRQHandler and read the time with a logic analyzer. Ensure it stays below your real‑time budget.
4. Use static analysis tools (Cppcheck, MISRA) to catch missing volatile qualifiers and potential overrun paths.

Conclusion & Action Items

Implementing a lock‑free circular buffer is straightforward once you:

• Choose a power‑of‑two size and align the buffer.
• Use volatile indices and enforce atomicity with brief interrupt masking or memory barriers.
• Detect full/empty by sacrificing one slot, avoiding an extra counter.
• Validate with assembly inspection, stress tests, and ISR latency measurement.

With these practices you’ll have a robust ISR‑to‑main data path that scales from simple UART echo applications to high‑throughput DMA ping‑pong pipelines.