Message Delivery Order Inconsistency in Waku Gossip Protocol

No, Waku does not provide a built-in mechanism to ensure deterministic global message ordering. Because Waku is a decentralized gossip-based protocol, messages propagate through a peer-to-peer mesh without a central sequencer. Consequently, different nodes may receive messages in different sequences based on network latency and topology.

The Cause of Ordering Inconsistency

The lack of ordering is a fundamental characteristic of the Waku gossip architecture. In a decentralized pub/sub model, messages take multiple asynchronous paths to reach their destination. While the store-and-forward mechanism ensures eventual consistency (meaning all nodes will eventually receive the message), it does not guarantee causal or sequential consistency. Factors contributing to this include:

• Variable Latency: A message sent later may take a shorter network path and arrive before an earlier message.
• Peer Churn: Nodes joining and leaving the network can shift propagation paths dynamically.
• Asynchronous Relaying: The relay layer focuses on availability and propagation speed rather than strict FIFO (First-In-First-Out) delivery.

Recommended Application-Layer Implementation

To achieve sequential consistency, you must implement ordering logic within the message payload itself. The following strategies are recommended based on your consistency requirements:

1. Sequence Numbering (Strict Ordering)

If a single publisher is sending a stream of data, embed an incrementing integer (sequence number) in each payload. The receiver should maintain a "last processed" counter and buffer any messages that arrive out of order until the missing sequence numbers are received.

2. Logical Clocks (Causal Ordering)

For multi-publisher environments where the order of interaction matters, use Lamport Clocks or Vector Clocks. This allows the application to determine if one message happened before another, regardless of the physical time of arrival.

3. Deterministic Timestamps (Loose Ordering)

While wall-clock timestamps are prone to clock drift across peers, they can be used for coarse sorting. For higher precision, combine timestamps with a unique publisher ID to break ties.

Verification Steps

To verify the behavior in your specific environment, you can run a simple sequence test:

1. Deploy three Waku nodes in different network zones.
2. Publish a series of messages (e.g., "Msg 1", "Msg 2", "Msg 3") from Node A.
3. Log the arrival order on Node C.
4. Observe if "Msg 2" occasionally arrives before "Msg 1" during periods of high network congestion.

Diagnostic Detail Needed: Are your messages originating from a single trusted publisher or multiple concurrent peers? The recommendation shifts from simple sequence numbers to vector clocks if multiple peers are updating a shared state.