Configure NATS JetStream for Durable Message Persistence

Problem: messages disappear on restart or consumer disconnect

With core NATS, publish is fire-and-forget. If a subscriber is down or the server restarts, messages are lost. JetStream adds durable storage to NATS Server and at-least-once delivery semantics through streams and durable consumers.

The useful takeaway is simple: put messages into a JetStream stream with file-backed storage, and consume with a durable consumer that acknowledges each message. The stream persists the data; the consumer state survives disconnects.

How JetStream provides durability

JetStream is the persistence layer built into NATS Server 2.0+. A stream is an append-only log of messages for one or more subjects. Messages are written to disk according to a retention policy and can be replayed to consumers.

A consumer is a named view over a stream. A durable consumer is identified by name and survives client restarts. Delivery guarantees are enforced by acknowledgments. With AckPolicy set to explicit, the server redelivers unacknowledged messages until the consumer acks or the message expires.

Key terms:

• Stream: durable log with retention, e.g., limits, interest, or workqueue.
• Consumer: subscription with delivery settings and state.
• Ack: client confirmation that processing succeeded.

Worked configuration: enable JetStream and create a durable stream

Enable JetStream on the server host. The server needs write permission to the store directory. Do not use a tmpfs mount for production.

# /etc/nats/nats-server.conf
jetstream {
  store_dir: "/var/lib/nats/jetstream"
  max_memory_store: 1GB
  max_file_store: 10GB
}

Start the server with JetStream enabled and verify the feature is active. Run on the server host as the service user.

nats-server -c /etc/nats/nats-server.conf

Check JetStream status from a client host with network access to the server.

nats server check jetstream

Expected check is that JetStream reports enabled and the store path is accessible. Risk: if store_dir is unwritable, JetStream will start in memory-only mode and lose data on restart.

Create a stream for an order subject with limits retention. Use a placeholder name for your stream.

nats stream add ORDERS --subjects "orders.*" --retention limits --max-msgs 1_000_000 --max-age 72h --storage file

This creates a file-backed stream that keeps up to 1 million messages or 72 hours, whichever comes first. Retention limits bounds disk use.

Create a durable consumer with explicit acks and a reasonable ack wait.

nats consumer add ORDERS WORKER1 --durable WORKER1 --ack-policy explicit --ack-wait 30s --max-deliver 5

The consumer name WORKER1 is durable. If the client disconnects, the server remembers the last delivered sequence and resumes from there on reconnect.

Publish example with a placeholder subject and payload. Run from any client with publish permission.

nats pub orders.created '{"id":"123"}'

Consume with explicit ack. The client library must call ack after successful processing. If ack is not sent within ack-wait, the message is redelivered up to max-deliver times.

Verify durability practically

A practical check for persistence is to confirm messages survive a server restart.

1. Publish a few messages to the stream.
2. List stream state to note message count.
3. Restart nats-server gracefully.
4. List stream state again and confirm the count is unchanged.

This confirms file-backed storage is working. Do not rely on in-memory inspection alone.

Limits and operational notes

JetStream storage is per NATS Server node. High availability is achieved with a JetStream cluster, not by replicating a single node. Clustering requires careful planning of storage and raft leadership. LeafNode federation is a different pattern for edge routing.

Disk-based persistence impacts latency and throughput under high write rates. Batching, appropriate max_memory_store, and SSD-backed store_dir reduce pressure. Memory limits must be set to prevent out-of-memory conditions when the write buffer grows.

Retention policy choice matters. Limits retention is bounded. Interest retention drops messages when no consumers exist. Workqueue retention removes a message once one consumer acks it, which is useful for fan-out vs work distribution.

Common mistakes

Forgetting to configure store_dir leads to memory-only operation. The server will start, but data is lost on restart. Always verify the path is writable and persistent.

Using NATS Streaming Server configuration for JetStream. NATS Streaming is a separate product with nats-streaming-server.conf and different APIs. JetStream is built into NATS Server.

Assuming at-least-once means exactly-once. Consumers must be idempotent because redelivery can happen after crashes or network partitions.

Setting ack-wait too short for slow processing causes unnecessary redeliveries. Set ack-wait higher than typical processing time and use max-deliver to bound retries.

Creating a non-durable consumer for a long-running service. Ephemeral consumers lose state on disconnect and will reprocess from the last acknowledged position of the stream, causing gaps or duplicates.

Ignoring disk space. Streams grow until limits are hit. Monitor store size and set max file store and retention to avoid uncontrolled growth.