Raspberry Pi Hardware Watchdog for Unattended Recovery

A headless Raspberry Pi that stops responding in a closet, a garage, or a field enclosure is a problem you cannot fix with SSH. The useful takeaway is not to prevent every hang, but to guarantee a defined recovery: the Pi’s built‑in hardware watchdog can force a reboot when the software stops servicing it, trading perfect visibility for predictable uptime.

The concrete problem: silent hangs on unattended Pi

Unattended deployments favor recovery over diagnosis. Network stacks can deadlock, a user process can pin the CPU, or a kernel oops can leave the system alive but unresponsive. Manual power cycling is not an option when the device is remote. A hardware watchdog provides a last‑resort reset that does not require a human.

On Raspberry Pi models with the Broadcom SoC, the bcm2835 watchdog driver exposes a hardware timer. When the timer is not periodically refreshed, or “kicked”, the hardware asserts a reset. The device appears in userspace as /dev/watchdog and is normally managed by a daemon that kicks it only when system health checks pass.

How Raspberry Pi OS exposes the watchdog

Availability depends on model and kernel. Pi 3, Pi 4 and Pi 5 are generally supported under 64‑bit Raspberry Pi OS with the bcm2835_wdt driver built in. The timeout window is set by kernel parameters and can vary across releases, so treat defaults as version‑sensitive.

Verification before changing anything:

• Run as root or with sudo on the Pi: ls /dev/watchdog. Presence of the node indicates the driver probed.
• Check driver load: lsmod | grep bcm2835_wdt. A line means the module is loaded.
• Inspect boot messages for probe information: dmesg | grep -i watchdog. Look for bcm2835_wdt probe messages.
• Check service state: systemctl status watchdog. Raspberry Pi OS ships a watchdog service that can be enabled.

Risk note: writing to /dev/watchdog without a proper daemon can trigger an immediate reboot. Only test kick paths in a controlled environment.

Worked example: kernel + systemd watchdog with a health gate

A practical pattern pairs the kernel driver with systemd’s watchdog support and a small health check. Systemd can monitor the service and write keepalives to /dev/watchdog when the system remains responsive.

Enable the kernel watchdog argument on the Pi, as root, by editing /boot/firmware/cmdline.txt on recent Raspberry Pi OS releases. Append a parameter such as watchdog or the specific timeout option supported by your kernel version. Reboot is required for kernel command line changes. This is a state‑changing operation; keep console access available for rollback.

Enable and start the userspace service:

sudo systemctl enable watchdog
sudo systemctl start watchdog

Expected check: systemctl is-active watchdog should report active. The service should hold /dev/watchdog open and refresh it on a regular interval.

For a health‑gated kick, create a systemd service that runs a check script and only touches the watchdog device on success. Example unit skeleton at /etc/systemd/system/pi-health-watchdog.service:

[Unit]
Description=Pi health gated watchdog kick
After=network-online.target

[Service]
Type=simple
ExecStart=/usr/local/bin/pi-health-kick.sh
Restart=always
RestartSec=30
WatchdogSec=60

[Install]
WantedBy=multi-user.target

The script /usr/local/bin/pi-health-kick.sh must be executable by root and must exit non‑zero on failure. Systemd will stop sending keepalives if the service fails, allowing the hardware timer to expire and reboot the Pi after the configured window. Do not invent output; verify behavior by inspecting service logs with journalctl -u pi-health-watchdog and confirming the watchdog device remains open.

Trade‑off and limitation

Automatic reboots improve availability but hide root causes. Intermittent faults become invisible without persistent logs, and a reboot during SD card writes increases wear risk and can corrupt filesystems. The watchdog also masks software bugs that would otherwise be debuggable.

Mitigations: use persistent logging to remote syslog or a read‑only root with journal on tmpfs, add application‑level heartbeats, and keep the watchdog timeout long enough to avoid spurious resets during normal load spikes. Timeout values are version‑sensitive; confirm the actual timeout for your kernel release in documentation before relying on it.

Rollback: disable the service with sudo systemctl disable --now watchdog and remove the watchdog kernel parameter from cmdline.txt, then reboot. This restores the default no‑watchdog state.

Actionable close

Check for /dev/watchdog and bcm2835_wdt presence first. Enable the systemd watchdog service only after you have remote logging and a way to recover the device if it reboots unexpectedly. Treat the watchdog as a recovery net, not a fix, and pair it with observability so reboots are explainable after they happen.