Filebeat close_older and close_inactive: Controlling Log File Harvesting

The Problem: Unbounded File Handles in Long-Running Filebeat Instances

Filebeat keeps a file handle open for every log file it harvests. On systems with high log rotation rates or many short-lived containers, this can exhaust file descriptor limits and memory. The close_older and close_inactive settings let you bound resource usage by closing files that are no longer receiving writes.

How the Two Settings Differ

close_older closes a file when its last modification time is older than the specified duration. It targets stale files that haven't been rotated or written to in a long time.

close_inactive closes a file after it has seen no new log lines for the specified duration. It targets files that have gone quiet but might still be open by the application.

Both settings only apply to new harvests started after the configuration reload. Files already being harvested when Filebeat starts or reloads continue until they naturally hit one of these thresholds.

Worked Configuration Example

The following snippet shows a typical setup for a Kubernetes node agent collecting container logs rotated daily:

filebeat.inputs:
- type: log
  paths:
    - /var/log/containers/*.log
  close_older: 72h
  close_inactive: 12h
  scan_frequency: 10s
  tail_files: true

close_older: 72h — stop harvesting any file whose mtime is older than 72 hours. This catches rotated logs that the application stopped writing to days ago.

close_inactive: 12h — if a file receives no new lines for 12 hours, close its handle. This handles containers that go idle but haven't been terminated.

scan_frequency: 10s — how often Filebeat scans the glob for new files. Shorter intervals pick up new containers faster but increase CPU.

tail_files: true — start reading at EOF for new files rather than from the beginning. Essential when close_older might re-open a rotated file.

Where to Apply This Configuration

Edit /etc/filebeat/filebeat.yml on the host running Filebeat (root or filebeat user with write access to the config directory). After saving, reload with:

systemctl reload filebeat

Or, if running in a container, update the ConfigMap and roll the DaemonSet. No downtime is required; Filebeat reloads inputs dynamically.

Verifying the Behavior

Run Filebeat with debug logging to see the close decisions in real time:

filebeat -e -d "harvester" 2>&1 | grep -E "close_older|close_inactive"

Look for lines like Close older file: /var/log/containers/app-123.log or Close inactive file: /var/log/containers/app-456.log. These confirm the thresholds are being evaluated.

Inspect the registry file to see which files are currently open and their last activity timestamps:

cat /var/lib/filebeat/registry/filebeat/data.json | jq '.[] | {path, offset, timestamp, ttl}'

The ttl field shows the remaining time before close_inactive would trigger. A missing or null ttl means the file is not subject to inactive closure (e.g., it's being tailed actively).

Confirm file handle reduction at the OS level:

lsof -p $(pgrep filebeat) | wc -l

Run this before and after the inactive period elapses. The count should drop as idle files are closed.

Limits and Common Mistakes

Already-Open Files Are Not Affected Immediately

If Filebeat is already harvesting 500 files when you add close_inactive: 1h, those 500 handles stay open until each file individually hits the inactivity threshold. The setting only governs future harvest decisions.

close_older Too Low Causes Incomplete Tailing

Setting close_older: 1h on a log that rotates every 30 minutes but keeps the same inode (common with copytruncate) will cause Filebeat to close the file before it finishes reading the rotated segment. The next scan picks up the new file, but the tail end of the previous segment is lost.

close_inactive with Truncation Risks Data Loss

If an application truncates its log file (e.g., echo "" > app.log) instead of rotating, Filebeat sees the file size shrink. If close_inactive fires before Filebeat reads the remaining bytes, those bytes are lost. Prefer log rotation that moves the old file aside.

Registry Persistence Across Restarts

The registry (/var/lib/filebeat/registry) persists file offsets and timestamps across restarts. If you change close_older from 72h to 24h, files harvested before the change still carry their original timestamps. They will not be re-evaluated against the new threshold until they are re-opened.

Practical Checklist Before Deploying

1. Measure current open file handles: lsof -p $(pgrep filebeat) | wc -l
2. Set close_inactive to 2–3× your longest expected application idle period.
3. Set close_older to slightly longer than your log retention policy (e.g., 72h for 48h retention).
4. Enable tail_files: true when using close_older to avoid re-reading rotated files.
5. Test in staging with filebeat -e -d harvester and verify close messages appear at expected intervals.
6. Monitor registry size: du -sh /var/lib/filebeat/registry — it should stabilize, not grow indefinitely.

When to Skip These Settings

If your Filebeat instance harvests fewer than a few hundred files total, or if every log file is actively written until rotation, the defaults (no automatic close) are fine. The settings add complexity and a small risk of missed lines; only enable them when you observe handle exhaustion or registry bloat.