Aerospike Multi-Record Transactions: When ACID Across Keys Actually Pays Off

The problem: atomicity across records without a distributed lock manager

You need to debit one account and credit another in a single atomic operation. In a traditional RDBMS you wrap both updates in a transaction. In Aerospike, single-record operations are atomic, but until version 4.6 (Enterprise) and 6.0 (Community) there was no built-in way to group multiple key-based reads and writes into one all-or-nothing unit. Teams either accepted eventual consistency, built custom compensation logic, or moved the invariant to an external coordinator. Multi-record transactions change that — but they come with a specific performance and operational profile you need to understand before adopting.

How it works: optimistic concurrency with version vectors

Aerospike transactions use optimistic concurrency control (OCC). When you start a transaction, the client reads the current record versions (generation counters) for every key you intend to modify. During the prepare phase, the server validates that none of those versions have changed since the read. If any record has been updated by another transaction or a non-transactional write, the entire transaction aborts with a conflict error. The client must then retry — typically with exponential backoff and idempotent logic.

Isolation is effectively serializable for the records participating in the transaction. However, non-transactional reads can see intermediate states: a record written inside a transaction becomes visible immediately after the prepare phase succeeds, before the final commit. If you mix transactional and non-transactional access on the same keys, you can observe partial updates. Design your access paths so that any code reading those records either always uses the transaction API or tolerates transient inconsistency.

Practical limits: record count, latency, and contention

The documentation and community benchmarks converge on a few guardrails:

• Record count: keep transactions under 10 records. Each additional key adds a network round-trip during prepare and increases the probability of a version conflict.
• Duration: aim for <100 ms end-to-end. The default transaction timeout is 1000 ms (configurable via transaction-max-ms on the server and TxnPolicy.txnTimeout on the client). Long-running transactions hold version locks longer, raising abort rates and potentially exhausting client thread pools.
• Contention: under low contention, expect 2–5× latency overhead versus single-record ops. Under high contention on the same keys, abort rates climb sharply and throughput collapses.

For high-throughput simple key-value access where single-record atomicity suffices, stick to batch reads/writes without transactions. Reserve multi-record transactions for invariant enforcement across related entities — debit/credit, inventory reservation + order creation, leaderboard updates with user stats.

Worked example: transfer with retry logic (Java client 6.x)

// Pseudocode — adapt to your error handling and logging
public void transfer(String fromKey, String toKey, long amount) {
    int maxRetries = 3;
    for (int attempt = 0; attempt < maxRetries; attempt++) {
        Txn txn = client.txn();
        try {
            // Read both accounts inside the transaction
            Record fromRec = txn.get(null, new Key(ns, set, fromKey));
            Record toRec   = txn.get(null, new Key(ns, set, toKey));
            
            long fromBal = fromRec.getLong("balance");
            if (fromBal < amount) throw new InsufficientFundsException();
            
            txn.put(null, new Key(ns, set, fromKey),
                    new Bin("balance", fromBal - amount));
            txn.put(null, new Key(ns, set, toKey),
                    new Bin("balance", toRec.getLong("balance") + amount));
            
            txn.commit();
            return; // success
        } catch (AerospikeException.TxnAbort e) {
            if (e.getResultCode() == ResultCode.TRANSACTION_CONFLICT) {
                // version conflict — retry after backoff
                Thread.sleep((long) Math.pow(2, attempt) * 10);
                continue;
            }
            throw e; // other abort reasons (timeout, etc.)
        } finally {
            txn.close(); // always release resources
        }
    }
    throw new TransactionFailedException("Max retries exceeded");
}

Key points: the Txn object batches reads and writes; commit() triggers the two-phase prepare/commit; TxnAbort with TRANSACTION_CONFLICT signals a version mismatch — the only error you should retry automatically. All other abort codes (timeout, namespace unavailable, etc.) indicate systemic issues.

Trade-offs you’ll hit in production

• Licensing: Community Edition gained transaction support in 6.0 (2022). If you’re on an older cluster, you need Enterprise Edition.
• Strong consistency required: the namespace must run with strong-consistency true and replication-factor >= 2. AP mode with transactions risks data loss on node failure because the commit protocol relies on the consensus quorum.
• No secondary indexes or scans: only key-based get, put, and delete participate. You cannot atomically update a record and a secondary index entry in the same transaction.
• XDR replication is per-record: cross-datacenter replication applies each write individually. A transaction that commits in the source cluster may be partially replicated if a failure occurs mid-stream — cross-region atomicity is not guaranteed.
• Schema changes: adding or removing bins on transactional records requires coordinated rolling upgrades. A version mismatch between client schema expectations and server record layout will surface as aborts.

Verify before you ship

1. Confirm server version: asinfo -v 'build' or asadm -e 'show version' — need 4.6+ (Enterprise) or 6.0+ (Community).
2. Check namespace config: asinfo -v 'get-config:context=namespace;id=' — verify strong-consistency true and replication-factor >= 2.
3. Run a test transaction with 2–3 records using your client SDK; measure commit latency and abort rate under synthetic load.
4. Monitor txn_commit_ok, txn_abort, txn_conflict via asinfo -v 'statistics' and correlate with client-side retry metrics.
5. Test failure injection: kill a node mid-transaction, verify rollback and data consistency on surviving nodes.

If abort rates exceed 1–2% under normal load, reconsider the transaction scope — either reduce the number of keys, shorten the critical section, or move the invariant to a single record with a compound bin (e.g., a map of account balances). Multi-record transactions are a sharp tool; use them where the correctness gain justifies the latency and complexity cost.