Harbor replication trigger limits: event-driven vs scheduled when jobservice workers are shared

Answer to Your Questions

1. How to bound replication worker usage so GC and scans aren’t starved?

Harbor does not expose a per‑policy worker quota. The only lever is the max_job_workers setting in the jobservice configuration. Increase that value or run an additional jobservice instance (if your deployment supports it) to give the pool more capacity during a large sync. There is no documented way to reserve workers exclusively for GC or scans.

2. Operational trade‑offs for a warm standby registry

• Event‑driven – new pushes are replicated almost immediately. Load scales with push traffic, so during quiet periods the workers sit idle. This keeps the standby fresh but can spike worker usage when a burst of pushes occurs.
• Frequent cron – replication runs at a predictable interval. The bulk work is batch‑processed, making it easier to schedule GC and scans in a low‑traffic window. However, the standby lags by the cron interval, and the job can still occupy workers for the duration of the run.

3. Mixing triggers on one policy

Yes. A single replication policy can have both an event‑driven trigger and a cron schedule (unless your Harbor version explicitly forbids it). This pattern is common: event‑driven for day‑to‑day freshness, plus an infrequent scheduled run that backfills missed events or performs a safety‑net sync.

Why the Worker Pool Matters

All asynchronous work in Harbor—replication, garbage collection, vulnerability scans—passes through the same jobservice. The jobservice starts a fixed number of workers (default 10 in many 2.x releases). When a large replication run starts, each replication task grabs a worker until it completes. Because there is no reservation mechanism, GC or scan jobs must wait until a worker becomes free, which can delay their start time.

Practical Steps for Your Deployment

1. Check the current worker limit:

   cat /etc/harbor/harbor.yml | grep max_job_workers
   

2. Observe behavior during a bulk sync:
   • Start a manual replication of a large project.
   • Open the Harbor UI task list and note how many replication tasks are queued vs running.
   • Check GC or scan jobs that start after the replication begins; see if they queue behind replication.
3. Adjust worker count if needed:

   # In harbor.yml
   jobservice:
     max_job_workers: 20
   

   Restart the jobservice and rerun the bulk sync to verify GC/scan latency improves.
4. Configure the replication policy:
   • Enable Event‑driven for new pushes.
   • Set a cron schedule (e.g., */30 * * * *) for periodic backfill.
   • Test by pushing a new image after enabling the policy and confirm a replication task appears.
5. Stagger GC/scan schedules:
   • Run GC during low‑push periods (e.g., 2 a.m.).
   • Schedule scans after the replication window to avoid overlapping heavy workloads.

Missing Diagnostic Detail

To refine the worker‑pool recommendation, please confirm the current value of max_job_workers in your Harbor deployment. This number directly influences how many concurrent replication tasks can run before GC or scans are delayed.

Key Takeaways

• Worker limits are controlled by jobservice configuration, not replication policy.
• Event‑driven triggers keep a standby registry fresh with load proportional to push traffic.
• Adding a cron schedule to the same policy provides a predictable backfill mechanism.
• Stagger GC and scans to avoid contention with large replication runs.