Reducing Memory Pressure with IAsyncEnumerable in .NET

The Memory Spike Problem

When building APIs that return large datasets, the standard pattern is to fetch a collection from a database, load it into a List<T>, and return it as a Task<IEnumerable<T>>. This creates a memory bottleneck: the server must allocate enough RAM to hold the entire result set before the first byte is ever sent to the client.

For a few hundred records, this is negligible. For tens of thousands of records, it leads to high memory pressure, frequent Garbage Collection (GC) pauses, and a slow Time to First Byte (TTFB) for the end user. The solution is IAsyncEnumerable<T>, which allows you to stream data as it is retrieved rather than buffering it all at once.

How Asynchronous Streaming Works

Introduced in .NET Core 3.0 and stabilized in .NET 5.0+, IAsyncEnumerable<T> enables a method to return items one by one asynchronously. Unlike a standard IEnumerable<T>, which blocks the thread during iteration, or a Task<List<T>>, which waits for the entire list to be populated, IAsyncEnumerable<T> uses a state machine to yield control back to the system between each item.

When combined with await foreach, the consumer processes the first item as soon as it is available, while the producer continues to fetch the next item in the background. This effectively flattens the memory profile of your request.

Implementation Example: Streaming Data from a Service

To implement this, use the yield return keyword within an async method. A critical detail is the [EnumeratorCancellation] attribute, which ensures that if the client disconnects, the server stops processing the stream immediately.

using System.Runtime.CompilerServices;

public class DataService
{
    // This method streams data instead of buffering it into a List
    public async IAsyncEnumerable<LogEntry> GetLargeLogSetAsync(
        [EnumeratorCancellation] CancellationToken ct = default)
    {
        for (int i = 0; i < 10000; i++)
        {
            // Simulate an async DB call or API request
            var entry = await FetchLogEntryFromDbAsync(i, ct);
            
            yield return entry;
        }
    }

    private async Task<LogEntry> FetchLogEntryFromDbAsync(int id, CancellationToken ct)
    {
        await Task.Delay(10, ct); // Simulate latency
        return new LogEntry { Id = id, Message = $"Log message {id}" };
    }
}

public record LogEntry { public int Id { get; init; } public string Message { get; init; } }

Integrating with ASP.NET Core Controllers

In .NET 5.0 and later, if your controller action returns IAsyncEnumerable<T>, the framework automatically handles the streaming. System.Text.Json will serialize the items as they are yielded, sending them to the client as a JSON array in chunks.

[HttpGet("stream-logs")]
public IAsyncEnumerable<LogEntry> StreamLogs(CancellationToken ct)
{
    return _dataService.GetLargeLogSetAsync(ct);
}

Trade-offs and Limitations

Streaming is not a universal replacement for lists. Consider these constraints:

• State Machine Overhead: For very small datasets (e.g., under 100 items), the overhead of creating the async state machine may actually be slower than simply returning a list.
• The N+1 Danger: If you are using an ORM like Entity Framework Core, be careful with lazy loading. Streaming a list of entities and then accessing a navigation property inside the await foreach loop can trigger a separate database query for every single item, destroying performance. Always use .Include() for eager loading.
• Connection Duration: Streaming keeps the HTTP connection open longer. In environments with aggressive load balancer timeouts, very slow streams may be terminated prematurely.

Verifying the Result

To verify that your application is actually streaming and not buffering, you can use a tool like curl with the --no-buffer flag:

# Run this in your terminal to see data arrive in real-time
curl --no-buffer http://localhost:5000/stream-logs

If the data appears incrementally on your screen, the stream is working. If there is a long pause followed by a sudden dump of the entire JSON array, the data is being buffered somewhere in your pipeline (likely by a custom middleware or a legacy formatter).

Rollback Strategy

If you encounter stability issues with streaming, you can revert to buffered loading by changing the return type to Task<List<T>> and using await _dataService.GetLargeLogSetAsync().ToListAsync() (via the System.Linq.Async package) to collect the results before returning.