Choosing MariaDB Galera for High Availability: An Architecture Note

What problem Galera actually solves

Galera is a synchronous multi-primary replication plugin for MariaDB. Every write is certified against the other nodes before commit, so a committed transaction exists on all nodes in the primary component. That gives you near-zero data loss on node failure and failover measured in seconds, because any surviving node can immediately serve traffic with current data.

The trade: certification adds latency to every write, conflicting writes on the same row abort one of the transactions, and the cluster can refuse writes entirely when it loses quorum. If your real requirement is read scaling or disaster recovery across regions, Galera is usually the wrong tool. If your requirement is \"a single node dying must never lose committed data or stop writes for more than a few seconds, within one site,\" it is a good fit.

Requirements to state before choosing it

• Zero or near-zero data loss (RPO ≈ 0) for committed transactions within one region.
• Failover fast enough that a connection retry succeeds — typically sub-second to a few seconds.
• All nodes in one low-latency network. Galera certification is sensitive to round-trip time; a rule of thumb is that commit latency grows by roughly one intra-cluster RTT.
• An application that can retry aborted transactions. Certification conflicts surface as deadlocks (typically error 1213), and hot-row contention makes them frequent.
• Mostly transactional InnoDB workloads. Large bulk loads, huge transactions, and non‑InnoDB tables replicate poorly or not at all.

If you cannot accept the retry requirement or the single-region constraint, stop here and consider asynchronous replication instead.

The smallest suitable design

Three nodes across at least two failure domains (racks, availability zones), all writable, with the application connecting through a proxy such as MaxScale or HAProxy. Three is the minimum that survives one node loss while keeping quorum. Two nodes plus a Galera Arbitrator (garbd) is a cost‑saving variant, but the arbitrator is itself a single point of failure for quorum — use it only when a third full node is genuinely unaffordable.

Writing to all nodes is supported, but in practice routing writes to one node at a time (with the others as hot standbys) sharply reduces certification conflicts, because the same row is less likely to be modified concurrently on two nodes. Reads can be distributed freely. This \"single-writer, multi-reader\" pattern keeps Galera's synchronous guarantees while avoiding its main pain point.

Key configuration decisions to make deliberately:

• wsrep_on=ON with the same wsrep_cluster_address on all nodes.
• wsrep_sst_method: mariabackup is the usual choice because it is non-blocking for the donor compared with rsync. Validate this against your MariaDB version — SST behavior is version-sensitive.
• innodb_flush_log_at_trx_commit=1 and sync_binlog=1 if your RPO-zero claim must survive OS crashes, not just MariaDB crashes.
• Bind the replication traffic to a dedicated network interface if available.

Trust and data boundaries

Galera assumes the intra-cluster network is trusted. Nodes authenticate to each other via SST credentials in the configuration, but replication traffic itself is plaintext unless you explicitly enable Galera's TLS options (socket.ssl_cert, socket.ssl_key, and related provider options). On any network you do not fully control, enable it.

Clients are untrusted and authenticate per node as with any MariaDB server — Galera does not change the client authentication model. Data-at-rest encryption is entirely outside Galera's scope; use InnoDB tablespace encryption and OS-level disk controls if you need it. Finally, and this is the boundary people most often get wrong: Galera is not a backup. A dropped table replicates synchronously and instantly to every node. You still need point-in-time backups from mariabackup plus binlog archiving.

Operational checks to wire up from day one

Run these on any node as a user with sufficient privileges (e.g., via mariadb -e from a monitoring host):

SHOW GLOBAL STATUS LIKE 'wsrep_cluster_status';   -- expect: Primary
SHOW GLOBAL STATUS LIKE 'wsrep_cluster_size';     -- expect: 3
SHOW GLOBAL STATUS LIKE 'wsrep_local_state_comment'; -- expect: Synced
SHOW GLOBAL STATUS LIKE 'wsrep_flow_control_paused';
SHOW GLOBAL STATUS LIKE 'wsrep_local_cert_failures';

Alert on: wsrep_cluster_status not equal to Primary (the node will reject writes), cluster size below expected, and wsrep_flow_control_paused sustained above roughly 0.1–0.2, which means one node cannot keep up and is throttling the whole cluster. A rising wsrep_local_cert_failures count means your application has write hotspots; check application logs for retry rates under load before blaming the cluster.

Also monitor intra-cluster network latency and packet loss directly. Certification stalls from a degraded network look like application slowness and are frequently misdiagnosed.

Failure modes to plan for

• Network partition: the larger partition keeps the Primary component and serves writes; the smaller side goes non-Primary and rejects them. With three nodes split 2/1, the single node is read-only. Design the application to handle write errors from a minority node.
• Certification conflicts: concurrent updates to hot rows abort one transaction. The fix is application-side: retry logic, shorter transactions, and single-writer routing.
• SST storms: a node rejoining after extended downtime requests a full state snapshot transfer, which copies the entire dataset from a donor. On large databases this saturates disk and network and can degrade the donor. Keep gcache.size large enough that routine restarts use incremental state transfer (IST) instead.
• Rolling upgrades: SST method behavior and wsrep provider compatibility vary between MariaDB versions. Test the upgrade path on a copy of the cluster; do not assume a donor running the old version can serve SST to a new-version joiner without checking release notes.

Conditions that should change the design

Revisit Galera if any of these become true: you need writes in more than one region (use asynchronous multi-source replication; Galera over WAN latency is painful), bulk-load throughput matters more than failover speed (certification overhead on multi-gigabyte transactions is prohibitive), or write conflict rates stay high despite single-writer routing (the workload may simply not suit multi-primary certification). In those cases a conventional asynchronous primary with a monitored replica and semi-sync plugin is simpler and cheaper to operate.

Verify the design under realistic load before production: measure commit latency, certification failure rate, and failover time by killing a node mid-load. Those three numbers tell you whether Galera fits your workload better than any documentation will.