Reboot loop during machine configuration application in Talos Linux

Answer – How to recover when the network is broken

A malformed machine configuration can lock a Talos node in a reboot loop because the gRPC server never starts. The OS intentionally removes a local shell, so manual intervention is only possible through a console that is always available (serial, KVM, or a local USB port). Talos provides two built‑in recovery pathways that work regardless of the network stack:

1. Debug container mode – a special container that exposes a shell and a local gRPC endpoint even if the node’s network is down.
2. Maintenance mode reset – wipes the current machine configuration and boots the node into a state where the gRPC server is disabled but the console remains reachable.

Step‑by‑step recovery using the debug container

1. Verify console access. From the host that can reach the node’s serial console, run:

   talosctl debug --node <node-address>

   If the node is already booted into maintenance mode, the container will start automatically. If the node is stuck in a reboot loop, the debug container will still start because it uses the same kernel but a different init process that does not require networking.

2. Check the local gRPC endpoint. Inside the debug shell, run:

   talosctl get node --node 127.0.0.1

   Success indicates that the local gRPC server is up. If it fails, examine the container logs for kernel or init errors.

3. Apply a corrected machine configuration. From the same host (or from the debug shell), push a minimal, valid configuration that sets a static IP or uses DHCP and a known SSH key:

   talosctl apply-config --force --node <node-address> --config minimal.yaml

   The --force flag bypasses the usual validation that would reject the broken configuration, allowing you to overwrite the corrupted state.

4. Reboot the node. After the configuration is applied, reboot normally:

   talosctl reset --node <node-address>

   This command clears the machine configuration and returns the node to maintenance mode. Once the node boots again, the new configuration will be applied automatically, and the gRPC server should start normally.

Alternative: Use maintenance mode reset directly

1. From a host with console access, run:

   talosctl reset --node <node-address>

   This wipes the machine configuration and boots the node into maintenance mode. The node will still expose a local gRPC endpoint and a debug shell.

2. Apply the corrected configuration as shown above, then reboot.

Key diagnostic question

Do you have a serial or KVM console attached to the node that is currently in the reboot loop? If not, you may need to attach a physical console or reinstall the Talos image.

Things to keep in mind

• Using --force skips validation; ensure the new config is syntactically correct.
• Resetting a node erases all stored secrets and certificates; a new cluster join may be required.
• The debug container image must be available on the node’s storage; if it’s missing, the node may fail to start the debug container.

Following these steps should break the reboot loop and restore normal operation of the Talos node.