Podman Quadlet and systemd restart policy conflict during healthcheck failures

The Arbitration of Restarts

Systemd does not prioritize a restart request based on a Podman healthcheck failure if the container process remains running. Because systemd monitors the process state (the PID of the container engine/shim) rather than the application state (the healthcheck status), a container marked as "unhealthy" by Podman is still seen as "active" by systemd. Consequently, systemd will not trigger a Restart= policy unless the container process actually exits.

Likely Explanation of Conflict

The ambiguity arises when both Podman's internal restart policies (e.g., --restart always) and systemd's Restart= directives are active. In a Quadlet-managed environment, the generated unit file is the primary authority. If a healthcheck failure leads to a process crash, both managers may attempt to intervene. This overlap often results in restart loops or race conditions where systemd kills the container to restart it just as Podman is attempting an internal recovery.

Recommended Configuration

To ensure systemd handles the lifecycle without conflict, delegate all restart logic to the systemd unit and disable Podman's internal policies. In your .container file, use the following approach:

• Remove any Restart= or --restart flags from the Container= or Exec= lines.
• Define the restart behavior in the [Service] section of the Quadlet file to ensure systemd is the sole orchestrator.

[Container]
Image=quay.io/example/app:latest
HealthCmd=curl -f http://localhost:8080/health

[Service]
Restart=on-failure
RestartSec=5s

Verification Steps

To verify that systemd is managing the recovery and not Podman's internal engine, use these commands:

1. Check the generated unit for the restart directive: systemctl cat container-app.service
2. Verify the container's internal policy is empty: podman inspect <container_id> --format '{{.HostConfig.RestartPolicy.Name}}'
3. Monitor the recovery sequence during a failure: journalctl -u container-app.service -f

Missing Diagnostic: Are you using a custom healthcheck script that explicitly calls exit 1 to terminate the container process upon failure? If the process does not exit, systemd will never trigger a restart regardless of the health status.