k3OS Node Bootstrapping: Cloud-Init Architecture and Trust Boundaries

The first-boot identity problem

A k3OS node has no interactive installer and an immutable root filesystem. The only practical moment to give it a cluster identity—the K3s server URL, cluster token, SSH keys, and agent flags—is the first boot, driven by cloud-init. If that data is missing, malformed, or unreachable, the node will not join the cluster and usually must be reprovisioned from scratch.

k3OS is a minimal, purpose-built operating system that runs only K3s. Its root filesystem is immutable; persistent data lives on a separate writable state partition. That design removes configuration drift, but it also means you cannot fix a bad bootstrap by logging in and editing a config file after the fact.

Requirements and the smallest suitable design

This note assumes a k3OS release that uses cloud-init for first-boot configuration. Because data-source support and key names change between releases, validate every detail against the release notes for your exact target version.

Data source and network reachability

Three requirements must be met before the node can register:

• A cloud-init data source must be present before the first boot completes: a NoCloud config drive, a cloud provider config drive, or provider user-data.
• The node must have network access to the K3s server during cloud-init execution so it can register and validate the cluster token.
• The user-data must carry the server URL, cluster token, SSH authorized keys, and any extra K3s server or agent arguments expected by that k3OS release.

A representative cloud-init payload

For a single edge site or lab, the smallest suitable design is one K3s server reachable at a stable address, plus a NoCloud config drive. The drive contains two files:

config-drive/
├── meta-data
└── user-data

The user-data file uses standard cloud-init syntax to install an SSH key for emergency access:

#cloud-config
ssh_authorized_keys:
  - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI... operator@workstation

The K3s server URL, cluster token, and any extra agent arguments are delivered through the k3OS-specific configuration mechanism documented for your target release. Because these keys differ across k3OS versions, confirm the exact field names in the release notes before generating production user-data.

Trust and data boundaries

Cloud-init user-data is plain text. On a NoCloud config drive it sits on a virtual disk; on a cloud provider it may be exposed through a metadata service. Anyone with hypervisor access, config-drive access, or metadata-service access can read the cluster token and SSH keys.

The cluster token is a bearer credential. Treat it like a password: use short-lived tokens where possible, avoid committing user-data to version control, and prefer encrypted delivery if your platform supports it.

The network path from the node to the K3s server is another boundary. Cloud-init needs that path open during first boot; a firewall or missing route will block registration. The writable state partition is local to the node, but cloud-init runs before the agent joins, so early boot is the only time identity is set.

Operational checks

After the node boots, confirm it appears in the server's Kubernetes node list. From the K3s server or a machine with kubeconfig access, run:

kubectl get nodes -o wide

Where: K3s server or admin workstation with cluster-admin kubeconfig.
Expected check: The new node appears with Ready status and the expected internal IP.
Risk: Read-only; no cluster state changes.

Inspect the node's boot logs to confirm cloud-init ran and the K3s agent started. On the node console or via serial log capture, look for cloud-init stage completion and K3s agent startup messages. If the node is unreachable over the network, the hypervisor's serial console is the fallback.

Check the K3s data directory on the node's state partition to verify the agent wrote its expected state. The exact path depends on the k3OS release; consult the release documentation.

Failure modes

• Unreachable server: if the K3s server URL is wrong or the network is down, cloud-init cannot validate the token. The node may boot but will not register.
• Invalid token: registration is rejected; the node remains absent from the node list.
• Unreadable state partition: if the writable state partition is missing or corrupted, the node cannot persist bootstrap state and may fail to start the agent.
• Version skew: each k3OS release is paired with a specific K3s version. Upgrading the OS without following the supported upgrade path can pair mismatched components and break the agent.
• Exposed credentials: plain-text tokens in user-data are a standing risk if the config drive or metadata service is not access-controlled.

When this design stops fitting

If you need to change cluster identity after deployment, this design is a poor fit. Post-boot changes to server URL or token usually require reprovisioning or a supported upgrade path.

If your security policy forbids plain-text secrets in user-data, evaluate whether your k3OS release supports encrypted config drives or an alternative secrets delivery mechanism before standardizing on cloud-init.

If nodes require ad-hoc package installation or system file edits, k3OS is the wrong base. Use a general-purpose Linux distribution and install K3s through configuration management instead.

Limitations and a practical verification path

k3OS is not a general-purpose distribution; installing packages or editing system files is outside the supported operating model. Cloud-init keys and behavior differ between releases, so configuration must be validated against the exact version in use.

A practical way to check the result is to boot a virtual machine with a NoCloud config drive containing a known server URL, token, and SSH key, then confirm the node appears in the server's Kubernetes node list with the expected K3s version. If it does not, inspect the boot logs for cloud-init errors and network reachability before reprovisioning.