Indexing Retried Events: Avoiding Duplicate Writes in Splunk Forwarders

Problem Statement

The goal is to guarantee that events sent from a Splunk forwarder are written to an index only once, even when transient network failures trigger retries. The forwarder can be configured with indexer_ack = true and a retry stanza, while the indexer may reject duplicates based on event IDs. However, it is unclear whether this combination provides true exactly‑once delivery or if duplicates can still slip through.

Key Constraints

• indexer_ack forces the forwarder to wait for acknowledgement before clearing its buffer.
• The retry stanza controls the number of resend attempts and back‑off strategy.
• Indexers can reject duplicates using duplicate_check, but the interaction with event IDs is version‑dependent.
• The dedup search command removes duplicates at search time, not ingestion time.

Unresolved Questions

1. Does enabling indexer_ack together with the default retry policy guarantee that a retried event is not written twice, or can duplicates still appear if the indexer acknowledges before dedup logic runs?
2. What is the current behavior of the indexer’s duplicate_check setting when indexer_ack is true, and how does it interact with the event’s event_id field across recent Splunk Enterprise releases?
3. Should administrators rely on search‑time dedup or configure indexer‑level duplicate detection to achieve exactly‑once semantics, and what trade‑offs exist between the two approaches?