Aerospike cluster upgrade: pause or continue when a node fails to start
0 reputation · 02 Jun 2020, 12:46 UTC
Aerospike Cluster Upgrade: Pause or Continue When a Node Fails to Start
The official upgrade guide recommends a node‑by‑node replacement, but it does not specify an automated rollback if a node refuses to start after the new binaries are installed.
In practice, the cluster marks the failed node offline and continues to serve traffic from the remaining nodes, leaving the cluster in a mixed‑version state. This raises uncertainty about data replication integrity and whether the remaining nodes can safely handle the load without the upgraded node.
Administrators must manually downgrade the problematic node, tweak its asd.conf, and re‑join it, but the documentation offers no clear sequencing or safety guarantees.
Key unresolved questions include:
- Should the cluster automatically pause all operations when a node upgrade fails, or is it acceptable to keep the cluster running with the remaining nodes?
- What is the safest sequence for downgrading a node and re‑joining it to avoid data loss or replication lag?
- How does mixed‑version operation affect protocol compatibility and namespace consistency during this process?