K3s SQLite vs embedded etcd: choose for failure domains, not just RAM

The decision hidden in the K3s install

The difficult moment in an edge deployment is when a simple one-server cluster becomes a multi-site or highly available system. K3s normally stores its state in SQLite, which keeps installation straightforward. Embedded etcd is available when the cluster needs replicated state and consensus. The useful takeaway is simple: choose SQLite for an independent, single-control-plane cluster; choose etcd when losing one control-plane node must not stop the cluster.

This is an architectural decision, not merely a resource optimization. Kubernetes API objects remain compatible with either backend, but their availability and scaling characteristics differ.

Requirement	SQLite	Embedded etcd
One K3s server	Straightforward fit	Works, but adds consensus overhead
Multiple K3s servers sharing state	Not supported	Supported through an etcd quorum
Control-plane node loss	Cluster remains unavailable until recovery	Can continue if quorum survives
Sustained API write churn	Single-writer behavior needs benchmarking	Better architectural fit for concurrent writes
Independent edge locations	One SQLite-backed cluster per location	Useful only for one logical cluster spanning locations

SQLite does not provide multi-server consensus. Do not configure two K3s servers to share one SQLite file, and keep the datastore on local storage rather than a shared network filesystem.

Worked example: three edge locations

Suppose a retailer operates one K3s server at a warehouse, one at a regional store and one at a distribution center. Each location runs local workloads and must continue operating when its internet connection fails. In this design, SQLite is a sensible choice: each location is an independent cluster with its own API endpoint and reconciliation process. A failure at one site does not require a distributed database to keep the other sites running.

Now suppose the requirement changes. The team wants all three locations to appear as members of one Kubernetes cluster, with a control-plane node that can be lost without making the API unavailable. SQLite cannot satisfy that requirement. The next candidate is embedded etcd with three etcd members. K3s can also run etcd-only nodes, separating the database role from workloads scheduled on servers.

Member placement then becomes an engineering decision. Etcd writes require agreement from a quorum, so a wide-area link can add latency and a partition can affect availability. Three members provide redundancy, but they do not make a partition harmless. Measure request latency from every location and decide which failure the cluster is expected to tolerate.

Write volume is a separate concern. Embedded etcd is generally the better fit for sustained write-heavy API traffic, while SQLite reduces operational complexity. Neither choice should be accepted on a synthetic benchmark alone: use manifests, controllers and admission behavior that resemble the production workload.

Verify the datastore before changing it

Identify the active backend before editing configuration or comparing performance. The following commands assume a standard Linux installation using the default /var/lib/rancher/k3s data directory. Run them on a K3s server as root, or as an account with read access to that directory.

sudo k3s --version
sudo test -f /var/lib/rancher/k3s/server/db/state.db && echo "SQLite datastore present"
sudo test -d /var/lib/rancher/k3s/server/db/etcd && echo "Embedded etcd data present"
sudo systemctl show k3s --property=ExecStart

For SQLite, expect the database file to exist and the embedded-etcd directory to be absent. For embedded etcd, expect the etcd data directory and relevant datastore or cluster initialization flags in the service configuration. Treat the startup configuration as the stronger evidence because files can remain after a configuration change.

Do not use the absence of an etcd pod in kube-system as the test. Embedded etcd normally runs inside the K3s process rather than as a separate pod. Also avoid pasting the complete service command into a public ticket, since it may contain a cluster token.

Record the installed version during this check. Paths and command-line flags can change between releases, so compare the result with the documentation for that specific version.

Check SQLite safely, then make the decision explicit

A structural check is useful after an unclean shutdown or suspected storage problem. It requires the sqlite3 command-line package. Run it on the K3s server as root, or as a user with read access to the database and write access to the temporary directory.

sudo install -d -m 0700 /var/tmp/k3s-db-check
sudo sqlite3 /var/lib/rancher/k3s/server/db/state.db ".backup '/var/tmp/k3s-db-check/state.db'"
sudo sqlite3 /var/tmp/k3s-db-check/state.db 'PRAGMA integrity_check;'

The expected result is ok. The backup command creates a consistent snapshot without manually copying the live database and its write-ahead log files. It can still contend with active writes, so schedule it away from the busiest period. The snapshot contains sensitive cluster state: keep the directory private and delete it after review.

A successful integrity check proves only that the copied database is structurally sound. It does not prove that certificates, tokens, workloads or an entire cluster can be recovered. A real recovery test must restore the data into an isolated environment. SQLite corruption can also require manual database intervention, making tested backups more important than the check alone.

Use a clear decision rule: choose SQLite for one control plane, independent edge clusters and write loads that pass a representative benchmark. Choose embedded etcd for multiple servers, quorum-based control-plane availability or sustained write pressure. Before production, exercise a node-loss scenario and record the recovery time and acceptable data loss. That turns “lightweight” from a slogan into an explicit service objective.