Systemd OnFailure=: Limits on Silent Alerting for Service Failures in Arch Linux

Configuring Silent Failure Alerts

To receive actionable alerts without desktop pop-ups, use the OnFailure= directive in your service unit to trigger a dedicated Type=oneshot service. This decoupled approach ensures that alerts are handled by system-level tools (like an MTA) rather than user-session notification daemons.

Reliability of OnFailure=

The OnFailure= directive is reliable for any unit that enters the failed state. This includes:

• Services that exit with a non-zero exit code.
• Services that time out during the startup phase.
• Services that crash due to signals (e.g., segmentation faults).

Assumption: This behavior assumes the service is managed by systemd. It will not trigger if a service is stopped manually via systemctl stop or if the process exits with code 0 despite a logical failure.

Configuration for Non-Desktop Alerts

To send emails or logs without a desktop session, you must use a system-wide Mail Transfer Agent (MTA) such as msmtp or postfix. User-level mail configurations are not accessible to system units.

1. Create the Alerting Unit: Create /etc/systemd/system/failure-alert.service

   [Unit]
   Description=Silent Failure Alert
   
   [Service]
   Type=oneshot
   ExecStart=/usr/bin/mail -s "Service Failure: %n" [contact removed] < /dev/null

2. Link to Target Service: Add the trigger to your application unit (e.g., /etc/systemd/system/myapp.service)

   [Unit]
   Description=My Application
   OnFailure=failure-alert.service

3. Apply Changes: Run systemctl daemon-reload.

Package Failures: Pacman Hooks vs. Systemd

For package installation failures, pacman hooks are the preferable approach. Systemd cannot natively monitor the internal state of a pacman transaction because these events occur outside the service lifecycle.

• Pacman Hooks: These are designed for package-related events. However, they typically trigger based on transaction stages, not necessarily on the failure of the pacman process itself.
• Dummy Services: Using a dummy service with OnFailure= is ineffective for standard package updates unless the update process is wrapped in a custom systemd unit.

Verification

To verify the setup, create a temporary service that executes /usr/bin/false and check the status of the alert unit:

systemctl list-units --state=failed

Confirm the alert was sent by checking the journal: journalctl -u failure-alert.service.

Note: Please specify if you are using a local MTA or a remote SMTP relay, as this changes the ExecStart configuration for the alert service.