K3s Embedded etcd vs External etcd: An HA Decision Guide

You provisioned a K3s cluster on three small servers, and the default embedded datastore worked perfectly—until you restarted a control plane node and the API server spent minutes reconciling state. For development or edge workloads, that is acceptable. For production, it is a signal to decide whether your cluster state should live on the same machines running your workloads or on a dedicated external etcd cluster.

The practical takeaway: embedded etcd keeps K3s simple and lightweight, but it couples the Kubernetes control plane to the compute nodes. External etcd decouples state and is the supported path for highly available control planes, at the cost of managing certificates, a load balancer or DNS endpoint, and a separate backup routine.

How embedded etcd behaves in K3s

K3s bundles a datastore by default. On a single server it typically uses SQLite, which avoids the memory overhead of etcd. On multiple servers with embedded etcd HA, K3s runs an etcd cluster across the control plane nodes using Raft consensus. This removes the single-node bottleneck but does not separate storage from compute.

Embedded etcd is not designed to scale beyond a small control plane. As cluster state grows, write latency and disk I/O on the server nodes compete with the Kubernetes scheduler and API server. A disk failure on a control plane node can put the entire cluster state at risk unless you have taken an etcd snapshot.

When external etcd is the better fit

External etcd moves the datastore to hosts that are not running K3s workloads. The K3s servers connect to the etcd endpoints over TLS, so you need valid client certificates and a reachable endpoint, usually backed by a load balancer or DNS record. This lets you restart, upgrade, or replace K3s servers without risking the cluster state stored in etcd.

It also means you are now responsible for two systems: the K3s cluster and the etcd cluster. You must monitor etcd disk latency, rotate certificates, and test snapshot restoration on a schedule.

Worked example: pointing K3s at an external etcd cluster

Run this on the first K3s server node with root privileges. Replace every placeholder with values from your etcd deployment.

# Run as root or with sudo on the first K3s server
k3s server \
  --datastore-endpoint="https://etcd-1.example.com:2379,https://etcd-2.example.com:2379,https://etcd-3.example.com:2379" \
  --etcd-ca=/etc/k3s/tls/etcd-ca.crt \
  --etcd-cert=/etc/k3s/tls/etcd-client.crt \
  --etcd-key=/etc/k3s/tls/etcd-client.key

After the server starts, verify the control plane is reachable and that K3s is not running a local embedded etcd instance:

# Run on the K3s server with kubectl configured
kubectl get nodes -o wide

# Confirm no local etcd process is bound to the default embedded port
ps aux | grep etcd

Expected result: nodes show Ready status, and ps returns only the grep command itself or etcd client connections, not a local embedded etcd server process. If nodes are NotReady or the API server logs show TLS handshake failures, check certificate validity, endpoint reachability, and whether a firewall blocks port 2379 between the K3s server and etcd hosts.

Trade-offs and migration limits

Switching from embedded to external etcd is a state-changing operation. You must take a snapshot of the embedded datastore, restore it into the external cluster, then redeploy K3s pointing at the new endpoint. Restoring a snapshot onto a different K3s or etcd version can fail because internal schemas change between releases.

Networking is another consideration. K3s defaults to Flannel, which does not enforce Kubernetes NetworkPolicy. If your HA design requires namespace isolation, you must install Calico or Cilium separately; changing the CNI after workloads are running typically requires draining nodes and reconfiguring pod CIDRs.

Rollback for migration: If the external etcd restore fails, keep the original embedded etcd control plane intact until you have validated the new cluster. You can redeploy K3s against the embedded datastore from a known-good snapshot, but only before production workloads have written new state to the external cluster.

How to verify the result

• kubectl get nodes -o wide — all control plane and worker nodes report Ready.
• ps aux | grep etcd — no local embedded etcd process on K3s server nodes when using external etcd.
• kubectl get pods -n kube-system — core system pods such as traefik and coredns are running.

If verification fails, do not force additional server nodes to join. Fix the etcd endpoint and certificate configuration on the first server before expanding the cluster.

Bottom line: Start with embedded etcd for single-node or small development clusters. Move to external etcd when you need control-plane high availability and can operate a dedicated etcd cluster with backups and certificate rotation.