Logstash Persistent Queue disk exhaustion and automatic purging triggers

Answer

Logstash does **not** automatically delete or purge Persistent Queue (.sqs) files when the underlying filesystem runs out of space. The only automatic cleanup occurs when a queue segment (page) has been fully acknowledged by downstream outputs; at that point the segment file is removed. If a new segment cannot be written because of ENOSPC (no space left on device) or because queue.max_size is reached, Logstash applies backpressure and stops reading from inputs instead of dropping data.

Confirmed facts

• Persistent Queue stores unacknowledged events as memory‑mapped segment/page files under path.queue/.sqs.
• Segment files are deleted only after they are fully acknowledged.
• There is no low‑disk‑space trigger that initiates a purge.
• When disk space is exhausted or the queue size limit is hit, Logstash prioritizes data retention over pipeline availability and exerts backpressure.

Likely explanation / behavior

Because the queue is designed for durability, Logstash prefers to stall the pipeline rather than risk losing unacknowledged events. If downstream consumers continue to acknowledge, already‑processed pages are removed automatically, which can free space once the filesystem becomes writable again.

Steps to recover

1. Check Logstash logs for no space left on device or ENOSPC messages around path.queue/.sqs writes.
2. Verify whether inputs are stalled (backpressure) by looking for “stopping input due to full queue” or similar lines.
3. Free disk space on the mount holding path.queue (delete unrelated files, expand volume, etc.).
4. If space is freed, restart Logstash; the pipeline should resume and automatically remove any fully acknowledged pages.
5. If the queue remains stuck after a restart (e.g., checkpoint files cannot be written), stop Logstash, then manually inspect path.queue/.sqs. Only delete files that you have confirmed are fully acknowledged (e.g., by checking the .sincedb checkpoint or by ensuring downstream has processed all events). Removing unacknowledged pages will cause data loss.

Missing diagnostic detail

To decide whether you need to free disk space or simply adjust queue.max_size/input rate, confirm whether the Logstash error is ENOSPC (filesystem full) or a backpressure message due to queue.max_size being reached. This detail changes the remedy: a pure queue‑size limit may be resolved by raising the limit or throttling inputs, whereas a true filesystem exhaustion requires space reclamation.