Resolving Upstream DNS and CA Trust in k3os
To persistently configure upstream DNS forwarders in k3os without them being overwritten by the node's resolv.conf, you must modify the CoreDNS ConfigMap to use explicit IP addresses instead of the default /etc/resolv.conf reference. To handle CA certificates on a read-only filesystem, you must utilize a HostPath mount or a ConfigMap to inject certificates into the pods, as the read-only root prevents direct modification of the host's trust store.
Persistent Upstream DNS Configuration
By default, CoreDNS in k3s/k3os uses the forward . /etc/resolv.conf directive. Because the k3s agent may synchronize the node's resolver settings, the most stable method is to hardcode the upstream providers in the CoreDNS configuration.
- Retrieve the current ConfigMap:
kubectl get configmap coredns -n kube-system -o yaml > coredns.yaml
- Locate the
forward . /etc/resolv.conf line in the Corefile.
- Replace
/etc/resolv.conf with your specific upstream DNS IPs (e.g., forward . 8.8.8.8 1.1.1.1).
- Apply the changes:
kubectl apply -f coredns.yaml
Note: To ensure these changes survive cluster-wide updates or k3os re-imaging, these modifications should be managed via a GitOps pipeline or a k3s manifest file located in /var/lib/rancher/k3s/server/manifests/.
CA Certificate Injection in Read-Only Environments
Since the k3os root filesystem is read-only, attempting to write to /etc/ssl/certs on the host will fail. The recommended approach is to propagate the trust anchor at the pod level.
- Option A (ConfigMap Mount): Store the custom CA certificate in a Kubernetes ConfigMap and mount it into the application pods at
/etc/ssl/certs/ca-certificates.crt (or the distribution-specific path).
- Option B (HostPath Mount): If the CA is generated during boot and stored in a writable partition (like
/var), mount that specific directory into the pod using a hostPath volume.
Verification Steps
Verify DNS resolution from within a pod using a scoped command:
kubectl exec -it <pod-name> -- nslookup google.com
Verify CA propagation by checking the certificate bundle inside the container:
kubectl exec -it <pod-name> -- ls /etc/ssl/certs/
Required Diagnostic Detail
To provide a more specific mount path for the CA certificates, please specify if your custom CA is stored in a persistent /var subdirectory or if it is provided as an external file during the k3os deployment process.
k3os