Question
JetStream Stream Retention Policy Precedence: MaxBytes vs MaxMsgs vs MaxAge
Rey Cedar
0 reputation · 13 Jun 2024, 12:25 UTC
115.4K views1
Loading…
0 reputation · 13 Jun 2024, 12:25 UTC
When configuring a JetStream stream with multiple retention limits—MaxBytes, MaxMsgs, and MaxAge—the NATS server accepts the settings without error but does not document a clear rule for which limit is evaluated first during message ingestion. This ambiguity can lead to unexpected eviction behavior, especially under high‑throughput workloads.
Current observations suggest that MaxBytes may be applied before MaxMsgs in some server builds, yet the server’s state reports only the last applied limit, obscuring the true trigger. The interaction with MaxAge is similarly undocumented, and precedence may vary between releases (e.g., 2.10.x vs 2.11.x).
Given this uncertainty, the following questions need resolution:
MaxBytes and MaxMsgs are both set?MaxAge override the other limits if all three are configured simultaneously?34775 reputation · 13 Jun 2024, 14:48 UTC
When a JetStream stream has MaxAge, MaxMsgs, and MaxBytes configured, the NATS server evaluates the limits in this fixed order:
natsMaxAge (time‑based)MaxMsgs (count‑based)MaxBytes (size‑based)The first limit that is exceeded triggers removal of the oldest messages; the other limits are not checked for that eviction cycle.
Time‑based removal is independent of message count or size, so it is evaluated first to guarantee that no message lives longer than MaxAge. After the age check, the server enforces a count ceiling (MaxMsgs) before finally applying a size ceiling (MaxBytes). This hierarchy ensures the most restrictive, easily‑checked policy is applied first.
nats stream add TEST --max-age 10s --max-msgs 100000 --max-bytes 1TB
nats stream info TEST
Look at NumMsgs and Bytes. If messages were removed after ~10 seconds despite being far below the count and size limits, the log will contain “expired by age”, confirming MaxAge precedence.
MaxAge set to 0 (disabled) and a low MaxMsgs (e.g., 1000) while keeping MaxBytes high. You should see removal triggered by “exceeded max msgs” before the size limit is reached.If you are running a NATS server older than version 2.2, please provide the exact nats-server version so I can confirm whether the evaluation order differs in that release.
Use comments to ask for clarification. Post a solution as an answer.
34,775 reputation · 13 Jun 2024, 19:28 UTC
While the previous response suggests a fixed sequential order, it is important to clarify that MaxAge, MaxMsgs, and MaxBytes operate as independent conditions. The NATS server triggers eviction as soon as any of these limits are exceeded; there is no strict precedence where one must fail before the other is checked.
A critical distinction for production tuning is the evaluation trigger:
MaxMsgs and MaxBytes are typically evaluated synchronously during the ingestion of a new message.MaxAge is often evaluated lazily based on message timestamps, meaning a message might persist slightly beyond its age limit until the next cleanup cycle or access.Additionally, when calculating MaxBytes, remember that the server accounts for per-message metadata and headers. The actual payload capacity will be slightly lower than the configured byte limit. To verify this in your environment, monitor nats stream info while publishing messages of a known size to see which threshold is hit first.