Clojure core.async: Why Blocking Calls in go Blocks Stall Your Thread Pool

The Problem: go Blocks Are Not Threads

If you call a blocking Java API such as Thread/sleep, a synchronous HTTP client, or a JDBC query inside a core.async go block, the entire asynchronous runtime can grind to a halt. The go macro does not create a new operating-system thread per block; it parks the block and resumes it later on a small, fixed-size thread pool. When every thread in that pool is blocked on I/O or a sleep, no other go block can make progress, even if those blocks are ready to run.

The useful takeaway: treat go blocks as lightweight coroutines that must never block. Use channels to coordinate work, and push genuinely blocking operations onto dedicated threads or a Java ExecutorService.

How Channels and go Blocks Work Together

A channel is a first-class queue that go blocks use to hand off values. You put a value onto a channel with >! and take one off with <!. Both operations park the current go block if the operation cannot complete immediately, freeing the underlying thread to run other blocks. This cooperative scheduling is what makes core.async workflows composable without the memory cost of one thread per task.

Channels can be unbuffered, which creates a rendezvous: the producer and consumer must meet at the same moment. More commonly, you attach a buffer so producers can run ahead of consumers within a bounded limit.

Worked Example: A Bounded Producer-Consumer Pipeline

The following example assumes a Leiningen project with [org.clojure/core.async "1.5.648"] in project.clj. The producer emits the integers 1 through 10, and the consumer prints them. A fixed buffer of size 2 lets the producer stay slightly ahead without unbounded memory growth.

(ns example.core
  (:require [clojure.core.async :as async]))

(defn producer [c done]
  (async/go-loop [n 1]
    (if (<= n 10)
      (do (async/>! c n)
          (recur (inc n)))
      (do (async/close! c)
          (async/>! done :producer-done)))))

(defn consumer [c done]
  (async/go-loop []
    (if-let [v (async/<! c)]
      (do (println "Received:" v)
          (recur))
      (async/>! done :consumer-done))))

(defn -main [& args]
  (let [c    (async/chan 2)   ; fixed buffer of 2
        done (async/chan 2)]
    (producer c done)
    (consumer c done)
    (async/<!! done)          ; wait for producer to finish
    (async/<!! done)))        ; wait for consumer to finish

Run this with lein run from the project root. Expected behavior: the numbers 1 through 10 appear in order, and the JVM exits cleanly. If the program hangs, the most likely cause is that the consumer go block never started or the channel was never closed.

Why the Buffer Size Matters

The 2 in (async/chan 2) creates a fixed buffer. When the buffer is full, the producer parks until the consumer makes room. This is back-pressure: the pipeline naturally slows down when the consumer is slow. Without a buffer, the producer and consumer would rendezvous on every value, which is safe but slower for bursty workloads.

Buffer Types and Their Trade-offs

core.async offers three buffer strategies. Choosing the wrong one is a common source of subtle production bugs.

Buffer	Behavior when full	Back-pressure	Typical use
Unbuffered (chan)	Producer and consumer rendezvous	Immediate	Strict hand-offs, request/response
Fixed (chan 10)	Producer blocks	Yes	Work queues, bounded pipelines
Sliding (chan (sliding-buffer 10))	Drops oldest element	No	Latest-value caches, telemetry
Dropping (chan (dropping-buffer 10))	Drops newest element	No	Sampling, load shedding

Sliding and dropping buffers never block the producer, which makes them useful for metrics or UI updates where stale data is worse than missing data. However, they silently discard work, so they are inappropriate for financial transactions or any pipeline where every message must be processed.

Common Mistakes and Limitations

Blocking Inside go Blocks

The most damaging mistake is invoking a blocking Java API inside go. Because the go thread pool is small, a handful of blocked threads can starve every other coroutine. If you must call a blocking library, wrap the call in clojure.core.async/thread, which runs on a cached thread pool, or submit the task to a Java ExecutorService and communicate the result back over a channel.

Unbounded Memory Growth

Large fixed buffers and deeply nested channels can retain references to objects long after they are useful. A fixed buffer of 100,000 items in a high-throughput pipeline may hold hundreds of megabytes of live data. Prefer small buffers and let back-pressure do its job.

Debugging Opacity

Parked go blocks do not appear in standard thread dumps, which makes deadlocks harder to diagnose. Enabling *print-dup* can help when logging channel values, and wrapping risky sections with alt!! and a timeout channel lets you detect a stuck operation instead of hanging forever.

Verification Checklist

• Correctness: Run the producer-consumer example above. Confirm that 1 through 10 print in order and the process exits.
• Back-pressure: Add a Thread/sleep inside the consumer go loop. With a fixed buffer, the producer should park once the buffer fills, not crash.
• Stall detection: Move that same Thread/sleep into the producer go block and wrap the take in alt!! with a timeout channel. If the timeout fires, you have confirmed the thread pool is starved.

None of these checks require special permissions beyond the ability to run a JVM locally. If you integrate core.async with an existing ExecutorService, verify that blocking tasks are submitted to the executor and that only non-blocking coordination happens inside go blocks.