Docker Pipeline Plugin: Uncertain Agent Provisioning on Low Traffic Jenkins

For a low‑traffic Jenkins workload, the Docker Pipeline plugin can spin up temporary agents on demand, but the choice of Docker host, cleanup strategy, and resource limits directly affect cost savings and build reliability. Below is a breakdown of the likely explanations, confirmed facts, and the minimal steps needed for this scenario.

Likely explanation

Selecting the underlying Docker host shapes the trade‑off between cost, isolation, and maintenance. A local Docker daemon requires no extra infrastructure and is the simplest path for sporadic builds, but it does not auto‑scale or share containers across nodes. Docker Swarm adds basic clustering and service discovery with minimal setup, offering moderate cost savings and isolation while retaining standard docker run semantics. Kubernetes provides the most scalable on‑demand agent lifecycle, automatic node pooling, and integrated container pruning via TTL‑based pod termination, but demands higher operational expertise. For most low‑traffic Jenkins environments, the local Docker daemon is the starting point; Swarm is a middle ground; Kubernetes is reserved when build volume or isolation requirements grow.

Confirmed facts

The Docker Pipeline plugin does not automatically prune unused containers across all Jenkins versions. Orphaned containers can accumulate and consume disk space if cleanup is not explicitly configured. Explicit post‑build cleanup using the plugin’s afterBuild or afterContainer steps, combined with docker system prune on the host or ttlSecondsAfterFinished on Kubernetes agent pods, reliably removes idle containers where supported. However, pruning behavior varies by Jenkins LTS version; older LTS releases may not expose the afterContainer hook, and aggressive resource limits can trigger build retries that offset cost savings.

Steps needed for this case

1. Verify the Docker Pipeline plugin version is ≥ 1.23 through Manage Jenkins → Plugins → Installed. This ensures the afterContainer hook is available on Jenkins LTS where it is supported.
2. Identify the Docker host type in use. For a local daemon, add an afterBuild step that runs docker system prune -f --filter until=24h or configure Jenkins workspace cleanup to remove the agent container after each build.
3. If using Docker Swarm, ensure the Swarm cloud configuration includes a reasonable containerTemplate and that the host runs periodic docker container prune or leverages Swarm’s built‑in service cleanup.
4. If using Kubernetes, set the ttlSecondsAfterFinished annotation on the agent pod template (e.g., ttlSecondsAfterFinished: 300). This automatically terminates pods after the build finishes and the specified idle timeout.
5. After a build idle period, run docker ps on the host or kubectl get pods --field-selector=status.phase=Succeeded to confirm no orphaned containers persist. If containers remain, the prune configuration for that Jenkins version needs adjustment.

Missing diagnostic detail

To determine whether the pruning configuration is the primary blocker for your setup, it helps to know the Jenkins LTS major version currently in use. This affects whether the plugin’s afterContainer hook is available and which prune commands are safe to automate.