Stop Unbounded Queue Growth: Bounded Channels in .NET

When the queue outgrows the worker

Your background worker is fast enough—until it isn't. An HTTP endpoint, a timer, or a message pump enqueues work faster than the worker can process it. If you reached for ConcurrentQueue<T> or an unbounded Channel<T>, the queue silently absorbs the difference. Under sustained load, a slow dependency—an HTTP call, a database write, a disk flush—turns into memory pressure. The process does not crash immediately; it just gets heavier until the GC thrashes or the OOM killer intervenes.

The fix is not Task.Run. The fix is telling the producer to wait when the buffer is full.

Bounded channels: a buffer that says "slow down"

System.Threading.Channels gives you a thread-safe producer/consumer buffer with built-in backpressure. Instead of an unbounded structure, create a bounded channel:

var channel = Channel.CreateBounded<WorkItem>(new BoundedChannelOptions(capacity: 100)
{
    FullMode = BoundedChannelFullMode.Wait,
    SingleReader = true,
    SingleWriter = false
});

With FullMode = BoundedChannelFullMode.Wait, a full channel makes WriteAsync wait instead of silently accepting more work. SingleReader and SingleWriter are optimizations, not safety switches. Set them to false when multiple consumers or producers may run concurrently.

Worked example: producer, consumer, and completion

Here is a minimal pipeline shape. The producer awaits WriteAsync in a loop; the consumer uses await foreach over ReadAllAsync(). When the producer finishes, it calls Complete().

// Producer
foreach (var item in items)
{
    await channel.Writer.WriteAsync(item, stoppingToken);
}
channel.Writer.Complete();

// Consumer
await foreach (var item in channel.Reader.ReadAllAsync(stoppingToken))
{
    await ProcessAsync(item, stoppingToken);
}

The consumer loop ends naturally after the channel is drained and completed. No sentinel values, no polling.

Completion is terminal. After Complete(), further writes fail and ReadAllAsync stops once drained. Do not reuse the same channel for a new batch.

Make cancellation and failure part of the design

If the consumer throws or the token is cancelled, the producer can hang forever waiting for space. Use TryComplete when ownership is ambiguous, and ensure exceptions or cancellation still lead to completion so a stuck consumer cannot hang the whole process.

try
{
    await foreach (var item in channel.Reader.ReadAllAsync(stoppingToken))
    {
        await ProcessAsync(item, stoppingToken);
    }
}
catch (Exception)
{
    channel.Writer.TryComplete();
    throw;
}

This unblocks producers waiting in WriteAsync.

Trade-offs: blocking, deadlocks, and drop semantics

FullMode.Wait gives real backpressure, but it can deadlock if the producer blocks a thread the consumer needs, or if callers use .Result or .Wait() around async channel APIs. Under a SynchronizationContext or during thread-pool starvation, a blocked producer can starve the very consumer that would unblock it. Prefer async end-to-end and keep the awaited work cancellable. Capacity is a tuning knob, not a correctness guarantee.

If dropping is acceptable, a non-wait FullMode can protect latency, but it changes semantics: writes may be rejected or items discarded. Count accepted versus dropped items to prove the intended behavior, and confirm the exact FullMode members for your target framework before relying on them.

How to check it works

Create a minimal console app. Run these commands in a terminal where the .NET SDK is installed; no elevated permissions are required:

dotnet new console -n ChannelBackpressure
cd ChannelBackpressure

Add a bounded Channel<int> with capacity 100. Run many producer items against a consumer that awaits Task.Delay(1). Confirm the program exits after Writer.Complete() without unbounded growth.

Then test shutdown paths: cancel the consumer token and throw inside ProcessAsync. Verify the producer is unblocked or completed and the process exits instead of hanging.

A bounded channel does not make the consumer faster. If ProcessAsync is slow because of a lock or sync-over-async, backpressure only moves the pain upstream. Fix the bottleneck first, then size the buffer to smooth bursts.