Provisioning k3os Nodes with Cloud-Config: Declarative Kubernetes Without the Bootstrap Scripts

You have a stack of bare-metal machines or VMs and you want a Kubernetes cluster on them. The usual path involves an OS install, a pile of shell scripts, SSH keys copied by hand, and a k3s install command you hope you ran with the same flags on every node. k3os was built to collapse all of that into a single YAML file you hand to the machine on first boot.

The useful takeaway: k3os treats a cloud-config file as the single source of truth for a node's identity and cluster role. Write it once, feed it to each machine, and every node comes up with the right hostname, SSH access, and k3s role — server or agent — with no post-install scripting.

What k3os actually does at boot

k3os is a minimal Linux distribution purpose-built to run k3s, the lightweight Kubernetes distribution from Rancher. Its root filesystem is read-only with an overlay, which means the OS itself is effectively immutable — you don't configure it by logging in and editing files, you configure it by declaring what you want up front.

On first boot, k3os runs a cloud-init style agent that reads a cloud-config YAML file. That file can define the hostname, authorized SSH keys, the k3s role (server for control plane, agent for worker), a cluster join token and URL, and any extra k3s arguments. The agent then installs and starts the appropriate k3s service automatically. Because the config is persisted and re-applied on every boot, the declaration stays authoritative — there's no drift between "what the node should be" and "what someone changed by hand last month."

A worked example: one server, one agent

Here is a minimal cloud-config for the first control-plane node. You'd place this on the install media or pass it via your hypervisor's user-data mechanism (for example, a cloud-init ISO attached to a VM):

# cloud-config for the first server node
hostname: k3s-server-1
ssh_authorized_keys:
  - ssh-ed25519 AAAA... your-key-here
k3os:
  k3s_args:
    - server
    - --cluster-init
  token: "replace-with-a-long-random-string"

And for a worker that joins that server:

# cloud-config for an agent node
hostname: k3s-agent-1
ssh_authorized_keys:
  - ssh-ed25519 AAAA... your-key-here
k3os:
  k3s_args:
    - agent
  server_url: "https://10.0.0.10:6443"
  token: "replace-with-a-long-random-string"

Replace the token with a genuinely random value and 10.0.0.10 with your server node's address. Both machines boot, apply their config, and the agent registers itself with the server.

To verify from any machine with cluster access (the kubeconfig lives at /etc/rancher/k3s/k3s.yaml on the server):

kubectl get nodes

You should see both nodes in Ready state. On the node itself, you can confirm the applied arguments by inspecting the generated k3s configuration under /etc/rancher/k3s/ (server) or the agent config directory under /var/lib/rancher/k3s/agent/.

Changing your mind later: the upgrade controller

Provisioning isn't a one-time concern. k3os includes an upgrade controller that lets you push an updated cloud-config to a node — say, a new k3s version or a changed role — and it applies the change with a controlled reboot. This keeps the declarative model intact: instead of SSHing in to tweak things, you update the declaration and let the node converge on it.

Trade-offs and honest limitations

A malformed cloud-config halts boot. If the YAML is invalid, the node stops and waits for console access. In a datacenter that's an inconvenience; on a remote edge box with no out-of-band console, it's a site visit. Validate your YAML before deployment and keep a known-good copy.

Role changes don't migrate workloads. Flipping a node from server to agent via the upgrade controller doesn't drain or reschedule anything for you. Cordon and drain the node with kubectl drain first, or you risk disrupting whatever was running there.

Project status matters. k3os was archived by Rancher and is no longer actively developed, so it receives no new fixes or features. The cloud-config pattern it popularized lives on in other immutable/minimal Kubernetes OS approaches, and existing k3os installs still work — but for a new production deployment today, treat that maintenance status as a first-class factor in your decision, and verify the current state of the project before committing.

Where to start

If you want to evaluate the approach, spin up two VMs from the k3os ISO, attach the two cloud-configs above, and confirm kubectl get nodes shows both Ready. That fifteen-minute experiment tells you most of what you need to know about whether declarative, immutable node provisioning fits your environment — and the same mental model transfers directly to whatever successor tooling you adopt.