ReadMeFeed / Community knowledge
Real questions. Useful conversations. Find the people who know your stack.
Learn how Node.js stream backpressure works, why ignoring the write return value can exhaust memory, and see a concrete example of safely streaming a large file through a gzip transform while respecting drain events.
This post explains how Elixir’s GenStage keeps streaming pipelines bounded with backpressure, walks through a file‑processing example, and lists trade‑offs and verification steps.
Learn how to use Akka Streams' materialized values to coordinate side effects like database writes while keeping backpressure intact, with a concrete Scala example and trade‑off analysis.
When streaming a large relational dataset through an Akka Streams Alpakka JDBC source, the goal is to bound memory and query cost while preserving back-pressure. Offset-based pagination is straightforward to express using .drop(offset).take(limit) , but deep offsets force the database to scan and discard preceding rows. Keyset pagination avoids that cost by
Diagnosing a Backpressure Bottleneck We need to determine why throughput drops and latency spikes in an Akka Streams pipeline. The suspect is a backpressure signal propagating from a downstream stage, potentially causing upstream stages to pause and queues to grow. Constraints include the default 16‑element buffer per stage, the configured overflow strategy,
Persistent queue behavior changes when a Logstash pipeline moves from the default in-memory queue to a persisted disk-backed queue. The documented change shifts memory pressure to disk, alters backpressure when queue.max_bytes is reached, and changes restart semantics to replay unacknowledged pages. Version assumptions matter because queue.type names and def
Goal The Jaeger Collector exposes a gRPC server whose concurrent stream capacity is controlled by the grpc-server.max-concurrent-streams setting. When bursts of span ingestion exceed this limit, the collector may queue or reject new streams, leading to intermittent connection pool exhaustion that manifests as client reconnection storms. Uncertainty However,
Goal Determine how the backpressure semantics introduced in DataStax Java Driver 4.x alter the detection and handling of connection‑pool exhaustion compared with the 3.x series. Constraints and Uncertainty Configuration keys, metric names, and default pool sizes differ between the two major driver versions. Server‑side overload, hot partitions, and GC pauses
Goal: Determine whether gulp should automatically terminate the pipeline when any plugin emits an error while processing a large dataset, considering the backpressure flow from gulp.src through transforms to gulp.dest. Constraints: gulp.src creates a readable stream of Vinyl objects that respects Node.js backpressure; gulp.dest buffers incoming objects and f
nodejs
elixir
akka
scala
logstash
jaegertracing
grpc
cassandra
gulp