Portainer log access for auto-removing failed containers: persistent logging driver or disable auto-remove?

Direct answer

To see logs in Portainer for containers that disappear on failure, you have two reliable options:

• Keep auto‑remove and switch the logging driver to journald. The systemd journal retains logs independently of the container lifecycle, so Portainer can display them after the container is gone.
• Disable auto‑remove (remove --rm or the rm flag on a restart policy) and stay with the default json-file driver. The stopped container and its json-file log files remain on the host, giving Portainer full UI access.

Switching to json-file while keeping auto‑remove does not work – the log files are deleted together with the container. Other drivers (fluentd, gelf, syslog, splunk, awslogs, …) forward logs outside Docker; Portainer’s UI cannot read them, so you would still need the CLI or an external log system.

Why these two approaches work

journald + auto‑remove

• Logs are written to the systemd journal, which persists after the container is removed.
• Portainer (Community Edition 2.19+) reads the journal via the Docker logging‑driver API and shows the same output as docker logs.
• Retention is controlled by /etc/systemd/journald.conf (e.g., MaxRetentionSec, MaxFileSec). Verify the journal keeps logs long enough for your debugging window.

Disable auto‑remove + json-file

• The container stays in Exited state; its log file at /var/lib/docker/containers/<id>/<id>-json.log remains readable.
• Portainer’s Logs tab works exactly like docker logs.
• Trade‑off: stopped containers accumulate. Clean them periodically with docker container prune or a scheduled job.
• Enable log rotation (--log-opt max-size=10m --log-opt max-file=3) to avoid filling the host disk.

Driver‑specific limitations in Portainer

• json-file and journald – full UI parity with docker logs (including historical logs).
• All other drivers – Portainer shows “No logs available” or similar; you must use the external log destination (Elasticsearch, Graylog, CloudWatch, etc.) or the CLI on the host where the driver writes.
• Real‑time WebSocket streaming only works while the container exists. For fast‑failing containers, only stored logs matter.

Quick verification steps

1. Check current driver:

   docker inspect <container> --format '{{.HostConfig.LogConfig.Type}}'

2. If using json-file with --rm, deploy a test container without --rm, let it fail, then open Portainer → Logs. Logs should appear. Clean up with docker container prune.
3. If you prefer to keep auto‑remove, redeploy with --log-driver=journald --rm. After failure, check Portainer Logs and also run

   journalctl -u docker CONTAINER_ID=<id>

   to confirm parity.
4. Confirm Portainer edition and version (UI → Settings → About). Community Edition 2.19+ supports journald log viewing.
5. Verify journald retention:

   journalctl --disk-usage

   and inspect /etc/systemd/journald.conf for MaxRetentionSec/MaxFileSec.

One diagnostic detail that changes the recommendation

Do you need to preserve the auto‑remove semantics (i.e., the container must disappear automatically after failure)? If yes, use journald. If you can tolerate stopped containers lingering until a cleanup job runs, disabling auto‑remove with json-file is simpler and avoids journald configuration concerns.