Backpressure in Akka Streams: How It Works, How to Configure It, and Common Pitfalls

What Backpressure Actually Does

In Akka Streams, backpressure is the mechanism that keeps the data flow in check when one part of the pipeline can’t keep up with another. Every stage in a stream is a reactive streams publisher or subscriber. When a subscriber signals it can handle n more elements, the upstream publisher only emits that many. If the downstream stalls, the upstream automatically pauses, preventing a buffer from growing indefinitely.

Why You Should Care

Without backpressure you risk two things:

• Memory exhaustion – an unbounded buffer can fill the JVM heap.
• Dropped data – if you use a drop strategy, you silently lose elements.

Proper backpressure guarantees that each stage processes data at a rate it can handle, making the system robust under load.

Concrete Example

Below is a minimal stream that emits numbers 1..1000, filters even numbers, and logs them. The sink simulates a slow consumer by sleeping 50 ms per element. We tune the buffer to 16 elements and drop the oldest element if the buffer overflows.

import akka.actor.ActorSystem
import akka.stream.{ActorMaterializer, ActorMaterializerSettings, OverflowStrategy}
import akka.stream.scaladsl.{Source, Flow, Sink}
import scala.concurrent.duration._

implicit val system = ActorSystem("BackpressureDemo")
// Configure the materializer with a small buffer
val matSettings = ActorMaterializerSettings(system)
  .withInputBuffer(initialSize = 16, maxSize = 16)
implicit val materializer = ActorMaterializer(matSettings)

val source = Source(1 to 1000)
val filterEven = Flow[Int].filter(_ % 2 == 0)
val slowSink = Sink.foreach[Int] { i =>
  println(s"Received $i")
  Thread.sleep(50) // simulate slow I/O
}

// Attach a buffer that drops the oldest element on overflow
val buffered = filterEven.buffer(16, OverflowStrategy.dropHead)

source.via(buffered).to(slowSink).run()

Running this stream shows that the source emits elements only when the sink is ready. If you monitor the logs, you will see occasional "Dropped" messages when the buffer is full, indicating that backpressure is working.

Key Points in the Code

• ActorMaterializerSettings.withInputBuffer sets the maximum number of elements that can sit in the materializer’s internal queue.
• The .buffer stage is optional; without it the stream would use the materializer’s buffer, but adding a dedicated buffer lets you choose a different OverflowStrategy.
• Backpressure flows from the sink up to the source automatically; no explicit flow control code is required.

Configuration Options

Backpressure is driven by two layers of buffers:

Layer	Default Size	Typical Use
Materializer buffer	16	General purpose; tuned via ActorMaterializerSettings.
Stage buffer (e.g., .buffer)	Depends on stage	Fine‑grained control for hot spots.

To change the materializer buffer globally:

val matSettings = ActorMaterializerSettings(system).withInputBuffer(32, 1024)
implicit val materializer = ActorMaterializer(matSettings)

Here the materializer will hold up to 1024 elements before applying backpressure. Adjust these values based on the memory footprint of a single element.

Typical Limits and How to Detect Them

• OutOfMemoryError – occurs when the buffer grows beyond the heap. Watch for java.lang.OutOfMemoryError in the logs.
• Drop warnings – Akka logs a warning when OverflowStrategy.dropHead or dropTail is triggered. Look for "Dropped" in the log.
• Deadlock or thread starvation – often caused by blocking calls inside a stage. Use mapAsync or a separate actor to offload blocking work.

Common Mistakes

1. Ignoring demand signals – Writing a custom stage that never calls pull() or push() breaks the reactive contract and can freeze the stream.
2. Unbounded buffers – Using buffer(Int.MaxValue, OverflowStrategy.dropHead) defeats backpressure and can still lead to memory issues if the element size is large.
3. Blocking inside stages – Calls like Thread.sleep or synchronous I/O block the dispatcher thread. Use mapAsync with a dedicated dispatcher.
4. Too many async boundaries – Each .async adds a context switch. Over‑using it can degrade performance; use only when you need thread isolation.
5. Wrong overflow strategy – Choosing fail without handling the exception will crash the stream. If you need resilience, prefer dropHead or dropTail and monitor drops.

When Backpressure Fails

Backpressure only works if every stage respects the Reactive Streams contract. If you integrate a third‑party library that publishes without considering demand, the stream will still block or crash. In such cases you should wrap the library call in a mapAsync that respects the Future completion before pushing the next element.

Practical Checklist

• Set a realistic materializer buffer size based on element size.
• Use mapAsync for blocking I/O and configure its parallelism.
• Monitor logs for "Dropped" or "Buffer overflow" warnings.
• Test with TestKit to verify that backpressure stops the source when demand is low.
• Avoid custom stages unless you’re comfortable with the publisher/subscriber API.

Conclusion

Backpressure is the heart of Akka Streams’ resilience. By configuring the materializer buffer, using appropriate overflow strategies, and avoiding blocking code, you can build pipelines that scale without risking memory or data loss. Always test with realistic load and monitor for the typical signs of backpressure in action.