Using Lua Coroutines for Lightweight Cooperative Concurrency

The problem: needing non‑blocking multitasking without OS threads

When scripting game logic, network probes, or data‑processing pipelines you often want many independent tasks that can pause and resume. Creating OS threads for each task is heavy, and Lua’s single‑threaded VM would block if you used a simple loop. Lua coroutines give a cooperative alternative: each coroutine owns its own stack but shares the same Lua state, so switching costs only a few dozen bytes.

How coroutines work in Lua

A coroutine is created with coroutine.create (or coroutine.wrap) and started with coroutine.resume. Inside the coroutine you call coroutine.yield to pause and return control to the caller. The VM saves the coroutine’s stack, resumes another, and later restores the saved stack when resumed again.

Key characteristics

• Memory cheap: each idle coroutine needs only a small C stack (typically ~40‑80 bytes plus Lua stack).
• Deterministic GC: Lua 5.4’s garbage collector treats coroutine stacks as regular objects, so pause times stay predictable.
• No true parallelism: coroutines run sequentially; a blocking C call or a tight Lua loop that never yields stalls the whole VM.

Worked example: producer‑consumer pipeline

The following script creates a producer that yields numbers 1‥1000 and a consumer that resumes the producer and prints each value. Run it with a standard Lua 5.4 interpreter (lua producer_consumer.lua). No special privileges are required.

-- producer_consumer.lua
local function producer()
  for i = 1, 1000 do
    coroutine.yield(i)   -- pause and return i to the consumer
  end
end

local prod = coroutine.create(producer)

local function consumer()
  while true do
    local status, value = coroutine.resume(prod)
    if not status then error(value) end   -- propagate errors
    if coroutine.status(prod) == "dead" then break end
    print("Received:", value)
  end
end

consumer()

When you run the script you should see lines like:

Received: 1
Received: 2
...
Received: 1000

If the output stops early or you see an error, check that the coroutine was not resumed after it died (the status check prevents that).

Verifying low memory overhead

To confirm that many idle coroutines stay lightweight, create 10 000 of them that immediately yield and measure memory before and after:

-- mem_check.lua
local before = collectgarbage("count")
local coros = {}
for _ = 1, 10000 do
  table.insert(coros, coroutine.create(function() coroutine.yield() end))
end
local after = collectgarbage("count")
print(string.format("Memory increase: %.2f KiB", after - before))

Run with lua mem_check.lua. On a typical Linux/x86_64 build you should see an increase of only a few KiB (e.g., 30‑80 KiB), confirming the per‑coroutine cost is tiny.

Trade‑off and limitation

The cooperative nature means any blocking operation—such as a synchronous socket recv that does not yield, or a long‑running Lua loop without coroutine.yield—will stall the entire Lua state. If you need true parallelism for CPU‑bound work you must combine coroutines with OS threads (via LuaJIT FFI or the lua_thread library) or offload work to external processes.

Actionable closing

Start by wrapping any iterative or event‑driven logic in a coroutine that yields at each logical pause point (e.g., after receiving a network packet, after processing a batch of rows, or after each frame in a game loop). Use coroutine.status to detect finished coroutines and avoid resuming dead ones. Measure memory with collectgarbage('count') in your target environment to ensure the overhead stays within your budget, and profile with a simple debugger or debug.traceback to see that yield/resume points are clear.