asbackup Snapshot Consistency in Aerospike 7.x Rolling Upgrades
0 reputation · 19 Sept 2022, 20:52 UTC
0 reputation · 19 Sept 2022, 20:52 UTC
Use Aerospike’s asbackup and asrestore utilities to roll back a cluster that has begun a rolling upgrade on a 7.x release.
During a rolling upgrade, the cluster continues to accept writes. The documentation does not state that asbackup acquires a read‑lock or freezes writes, so the backup may capture a mix of old and new data. Operators must decide whether the backup is a true point‑in‑time snapshot or whether additional coordination is required.
asbackup support an internal lock or pause mechanism that guarantees a consistent snapshot during a rolling upgrade?33775 reputation · 20 Sept 2022, 06:59 UTC
No. asbackup does not acquire a cluster-wide read lock, freeze writes, or provide multi-record snapshot isolation. It is a logical record scanner that reads partitions sequentially over time. During a rolling upgrade, continued writes and node restarts mean the backup can capture a mix of pre- and post-upgrade records.
Aerospike guarantees consistency per record or transaction, not across the many partitions a full-namespace backup reads. If a node restarts or migrates mid-run, or if the application writes between partition scans, the output file contains records from different logical points in time. There is no supported configuration flag in asbackup that changes this behavior.
To approximate a single stable state, stability must come from operational quiescence rather than a backup flag:
asbackup run to a new output location. Do not append to or reuse an existing backup directory.The backup format is record-oriented and schemaless. It stores the bins present on each record at read time; it does not capture or enforce a namespace schema. If the upgrade introduces bin additions, removals, or type changes, the backup may contain heterogeneous records. On restore, asrestore writes records back as captured. Secondary indexes and UDFs are not part of the record backup and must be recreated separately on the target cluster. Validate that every bin type and feature is supported by the rollback target; otherwise the restore may fail, skip records, or produce data the older application cannot read.
For rollback, restore the pre-upgrade backup into a clean cluster running the target version rather than attempting an in-place reverse upgrade. A backup taken after the upgrade began is unsafe as a reverse-upgrade point if any node already accepted writes using new-version-only data features.
What are the exact source and target 7.x versions in your upgrade path? Mixed-version support and downgrade behavior are version-sensitive, and the answer determines whether a clean restore is mandatory or whether an in-place rollback is even possible.
Before relying on any backup, confirm the utility and server versions:
asbackup --version
asrestore --version
asinfo -h <host> -p 3000 -v build
Inspect asbackup --help on the exact binary for any lock, freeze, or snapshot options; their absence confirms the no-guarantee behavior described above.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.