Configuring Strong Consistency in Aerospike with COMMIT_ALL

Useful answer: enable strong consistency for writes

Aerospike can guarantee that a write is acknowledged only after it has been durably stored on every replica when you set the write policy commit level to COMMIT_ALL and configure a replication factor of at least two. This eliminates the chance of losing the write if a node fails immediately after the acknowledgment.

How it works: server and client settings

The server must store multiple copies of each record. The namespace definition controls the replication factor and storage engine. The client write policy tells the server to wait for all replicas to persist before returning success.

Namespace configuration (aerospike.conf)

namespace test {
    replication-factor 2
    storage-engine memory
}

With replication-factor 2 Aerospike maintains a primary and a secondary replica for each record in the test namespace.

Client write policy (Java)

WritePolicy wp = new WritePolicy();
wp.commitLevel = WritePolicy.CommitLevel.COMMIT_ALL;
Key key = new Key("test", "demo", "user:123");
Bin bin = new Bin("counter", 1);
client.put(wp, key, bin);

The commitLevel field forces the server to delay the acknowledgment until both the primary and secondary replicas have written the record to their storage medium (memory or SSD).

Worked example: end‑to‑end flow

1. Start a two‑node Aerospike cluster with the namespace above.
2. Run the Java client snippet from any host that can reach the cluster.
3. The client sends the write request to the primary node.
4. The primary writes the record to its storage and forwards the write to the secondary.
5. Both nodes persist the record; only then does the primary return a success response to the client.
6. If either node is unavailable or cannot persist within the client timeout, the write fails with a timeout error.

Limits and trade‑offs

• Increased latency: The client waits for the slowest replica, so write latency rises roughly proportional to replica synchronization time.
• Reduced throughput: Because each write consumes more network and disk I/O, the maximum sustainable write rate drops.
• Availability requirement: All replicas must be online; a single replica failure causes writes to fail until the node recovers or the cluster re‑balances.
• Resource usage: Each replica stores a full copy of the record, doubling (or more) memory or SSD consumption for the namespace.

Monitor the commit_latency metric in Aerospike Tools or the UI; a sustained increase indicates replicas are lagging and may affect overall latency.

Common mistakes and how to avoid them

• Forgetting to set the commit level: If the client policy leaves commitLevel at the default (COMMIT_MASTER), writes are acknowledged after only the primary persists, providing no strong‑consistency guarantee.
• Using replication‑factor 1: With a single replica, COMMIT_ALL behaves like COMMIT_MASTER because there is no secondary to wait for; the setting gives no extra safety but still incurs the latency of waiting for the (non‑existent) secondary acknowledgment.
• Mixing strong writes with eventual reads: A read performed with the default read policy (COMMIT_MASTER) may return a stale value if the secondary has not yet applied the write, even though the write was strong. Use ReadPolicy.CommitLevel.COMMIT_ALL for reads that must see the latest strong‑consistent write.
• Ignoring capacity planning: Enabling strong consistency on a memory‑backed namespace without accounting for the replica multiplier can cause out‑of‑memory errors. Verify available memory with asinfo -v "nodes" and ensure memory-size accommodates replication-factor * expected record size * record count.

Practical verification steps

1. Write a record using the client snippet above.
2. From a different client process, issue a read with ReadPolicy commitLevel = ReadPolicy.CommitLevel.COMMIT_ALL.
3. Confirm the returned bin value matches the written value.
4. Check server logs for lines containing commit-level: COMMIT_ALL and verify the write appears in the commit histogram of the Aerospike UI.

If the read returns the expected value and the logs show the commit level, the strong‑consistency path is functioning as intended.