Tuning Waku Relay Message TTL and Cache Size for WAN Delivery

The Problem: Messages Disappear, or Relays Run Out of Memory

You deploy Waku nodes across regions, and messages published in one data center never reach subscribers in another. Or worse, your relay nodes crash because their message caches grow without bound. Both symptoms point to the same engineering decision: how long a Waku relay should keep a message, and how many messages it should hold.

Waku uses relay nodes—libp2p peers that store and forward messages for topics they subscribe to—so a publisher does not need a direct path to every subscriber. This store-and-forward pattern helps messages cross WAN segments, but each cached message consumes memory, and every relay applies a TTL (time-to-live) after which the message is discarded.

How Waku Relay Caching Works

When a relay node receives a message on a subscribed topic, it caches the message and forwards it to other relay nodes or directly connected peers. If the intended recipient is offline, the message waits in the cache until either the recipient connects or the TTL expires. Rate limiting caps how many messages a relay accepts per second to protect against floods.

Because Waku implementations can differ, the exact configuration keys and default limits vary. Treat the following as an architectural pattern, not a copy-and-paste specification.

Worked Example: Sizing a Regional Relay

Imagine two offices connected by an unreliable WAN link that can drop for up to ten minutes. Your application publishes about two messages per second to the topic '/my-app/1/alerts'. You want messages to survive a ten-minute outage without exhausting relay memory.

# Illustrative Waku relay configuration: keys vary by implementation
relay:
  transport: wan
  topics:
    - '/my-app/1/alerts'
  cache:
    maxMessages: 2000
    messageTTLSeconds: 900
  rateLimit:
    messagesPerSecond: 10

The math is straightforward: two messages per second over 900 seconds means roughly 1,800 messages in the cache. Setting maxMessages to 2,000 gives modest headroom. A TTL of 900 seconds (15 minutes) exceeds the expected ten-minute partition, so messages should survive typical outages.

Run verification on the relay host as the service user that owns the Waku process. Use your implementation's metrics or log endpoint to check cache evictions and delivery delay; do not assume a specific CLI command exists. For example, look for log lines indicating message expired or cache evicted after a controlled test.

Expected check: publish a test message, disconnect the subscriber for five minutes, reconnect, and confirm delivery. Then disconnect for twenty minutes (longer than TTL) and confirm the message is no longer delivered. Risk: setting TTL too high increases memory usage and stores stale messages; setting it too low loses messages during routine partitions.

Trade-offs and Limitations

• Message ordering is not guaranteed. A message forwarded through multiple relays can arrive out of publish order. Applications needing strict sequencing must add their own sequence numbers.
• Implementation variation. Rate limiting and expiration behavior may differ between Waku implementations, so mixed relay networks can behave inconsistently.
• Memory pressure. Large caches improve delivery during long partitions but can trigger out-of-memory kills on constrained hosts.

Practical verification: monitor the relay's cache size and eviction rate under load. If evictions climb before TTL, either increase maxMessages or lower the publish rate.

Actionable Closing

1. Measure your typical publish rate per topic and your worst-case network partition duration.
2. Set TTL to at least 1.5 times the partition duration.
3. Set maxMessages to publish rate multiplied by TTL, plus 20 percent headroom.
4. Run a controlled offline test in staging before changing production relays.

There is no universal correct TTL. The right value depends on your network's outage profile and your relay's memory budget. Verify against your specific Waku implementation's documentation before deploying.