Using Raspberry Pi's Built‑in Hardware Watchdog for Automatic Recovery

Problem

A Raspberry Pi running a long‑lived service can become unresponsive due to a software deadlock, infinite loop, or uncontrolled resource consumption. Without operator intervention the board stays stuck until power is cycled, which is unacceptable for remote or unattended deployments.

Takeaway

Enable the kernel’s bcm2835-watchdog driver, configure a 10‑15 second timeout, and run the watchdog daemon to feed the hardware timer. If the daemon stops feeding, the SoC resets automatically, providing a lightweight recovery mechanism that needs no extra wiring.

Requirements

• Detect software hangs and trigger a full SOC reset without manual power‑cycle.
• Timeout must be short enough to catch faults but long enough to survive normal boot, kernel module loading, and occasional load spikes.
• No additional hardware; rely solely on the SoC’s integrated watchdog.
• Restrict access to the watchdog device to privileged users only.

Smallest Suitable Design

Kernel driver

The bcm2835-watchdog driver is included in the default Raspberry Pi kernel. It exposes /dev/watchdog and implements the hardware timer that resets the SoC when the timeout expires.

Userspace daemon

The watchdog package provides a daemon that periodically writes a single byte to /dev/watchdog. As long as the write succeeds within the timeout, the timer is reset.

Configuration

# /etc/watchdog.conf
watchdog-device = /dev/watchdog
max-load-1 = 24
min-memory = 1
watchdog-timeout = 15
# Optional: check a specific service
pidfile = /var/run/myapp.pid

The watchdog-timeout value of 15 seconds is a safe starting point; adjust after load testing.

Enabling the service

Run the following commands as root (or via sudo) on the target Pi:

# Install the daemon (if not present)
apt-get update && apt-get install -y watchdog
# Ensure the driver is loaded
modprobe bcm2835-watchdog
# Enable and start the daemon
systemctl enable watchdog
systemctl start watchdog

Required permission: root (or a user in the video group, which the daemon runs as).

Trust / Data Boundaries

• The device node /dev/watchdog is owned by root:video with mode 0660. Only root or members of the video group can open it for writing.
• The kernel enforces that only privileged processes can feed the timer; a compromised unprivileged user cannot disable the watchdog.
• The hardware timer runs independently of the CPU and memory subsystem, so user‑space corruption cannot stop it from counting down.

Operational Checks

Verify driver load

dmesg | grep -i bcm2835-watchdog
# Expected: "bcm2835-watchdog: loaded"

Confirm daemon activity

systemctl status watchdog
# Should show "active (running)" and periodic "watchdog: heartbeat" messages in journal.

Test reset behavior (perform on a non‑critical board)

1. Stop the daemon: systemctl stop watchdog
2. Watch the board (via serial console, SSH, or a GPIO‑driven LED). After approximately the configured timeout (e.g., 15 s) the Pi should reboot automatically.
3. After reboot, verify that the daemon is restarted by systemctl (enabled at boot).

Risk: If the timeout is too short, normal boot or a temporary load spike may trigger an unwanted reboot. Validate the value under realistic workload before deploying.

Failure Modes

• Daemon crash or hang: No feeds → watchdog times out → reset. This is the intended safety path.
• Timeout set too low: Spurious resets during boot, module loading, or brief CPU spikes. Mitigate by measuring boot time and peak load, then raising timeout accordingly.
• Kernel panic: The panic may prevent the daemon from feeding, leading to a reset – still acceptable. However, if the panic corrupts the watchdog driver itself, the hardware timer may not function; this is extremely unlikely for the BCM2835 block.
• Power loss: The watchdog cannot reset the board if power is removed; complement with a UPS or external power‑monitor for critical data integrity.
• SD‑card corruption: The watchdog does not protect against filesystem damage; use a read‑only overlay or periodic fsck for resilience.

Conditions That Would Change the Design

• If the application requires a timeout longer than the maximum supported by the BCM2835 watchdog (approximately 15 seconds), an external watchdog IC or a second‑stage supervisor would be needed.
• In environments where power interruptions are frequent and data loss must be avoided, pairing the hardware watchdog with a battery‑backed real‑time clock and a journaled, read‑only root filesystem becomes necessary.
• When regulatory certification demands independent verification of the reset circuit, adding an external watchdog with a separate power domain provides the required isolation.

Rollback (if you need to disable the watchdog)

Because enabling the watchdog changes system state, you can revert the configuration:

systemctl stop watchdog
systemctl disable watchdog
apt-get purge -y watchdog
# Optionally blacklist the driver to prevent automatic load
echo "blacklist bcm2835-watchdog" > /etc/modprobe.d/blacklist-watchdog.conf
update-initramfs -u

After reboot, verify that /dev/watchdog is no longer accessible and that the board does not reset when the daemon is stopped.