ReadMeFeed / Community knowledge
Real questions. Useful conversations. Find the people who know your stack.
Developer tools · #logstash
When Logstash stalls, the output stops while inputs keep running. This guide walks through diagnosing backpressure, checking queues, worker threads, heap, and disk I/O, then applies fixes like tuning the persistent queue, adjusting workers, and scaling horizontally.
Persistent queues turn Logstash from an in-memory pipe into a durable on-disk buffer. Here is how they work, how to configure them, and how to verify they actually prevent data loss during a crash.
Logstash's Dead Letter Queue captures events that outputs reject—mapping errors, schema mismatches, 4xx responses—so they can be inspected and replayed instead of silently dropped. Enable it per pipeline, rotate segments, and add a file-watcher for alerting.
Logstash appears alive but stops emitting events while the persistent queue grows. This guide walks through ordered checks, root causes, and targeted fixes for stalled pipelines.
Logstash guarantees event order only when running with a single worker. This blog shows how to configure a single worker, the impact on throughput, and how to verify ordering with a simple example.
A crash can erase events waiting in Logstash's default in-memory queue. Persistent Queues move that buffer to disk. This architecture note covers the smallest safe configuration, trust boundaries, operational checks, and failure modes.
Guide to deciding when to enable Logstash persistent queues for at‑least‑once delivery, with configuration steps, trade‑offs and validation.
Use Logstash Persistent Queues to keep events on disk when the process crashes, preventing data loss. Learn how to enable, verify, and monitor PQ, plus the trade‑offs and pitfalls.
Learn how to enable Logstash persistent queues, configure them, and verify they survive a crash without losing events.
elasticsearch