GitLab CI/CD Jobs Stuck in Pending: Diagnostic Guide

Problem and takeaway

A GitLab CI/CD pipeline creates jobs, but the jobs remain in Pending. They do not start, fail, or cancel, and no runner appears to have picked them up. Retrying the job or pipeline usually does not change the state.

Takeaway: most Pending jobs are caused by a mismatch between what the job requests and what runners can provide, not by a broken GitLab instance. Check tags, runner scope, protected-ref settings, and capacity before assuming infrastructure failure.

Recognizable condition

• Pipeline is created and jobs show Pending.
• Retrying the job or pipeline does not move it to running, failed, or canceled.
• No runner is shown as having picked up the job.
• The condition persists across multiple pipeline runs.

Likely causes and where to check

Ordered checks

1. Compare job tags with runner tags

   Open the stuck job in the project Pipelines or Jobs view and note its required tags. Then open the Runners area for the instance, group, or project and list each candidate runner's tags. GitLab matches tags exactly; a job tagged production linux will not run on a runner tagged only production.

   Expected check: every job tag appears on at least one active runner scoped to the project.

2. Confirm runner state and scope

   Check whether the runner is active, paused, or offline, and whether it is registered at instance, group, or project scope. A runner scoped to a different group or project cannot pick up the job. A paused or offline runner receives no jobs.

   Where: GitLab UI > Runners. Permissions: Maintainer or Owner for project runners; Owner or Administrator for group and instance runners.

   Expected check: at least one runner is active and not paused, with scope that includes the project.

3. Compare pipeline ref with runner protected flag

   If the pipeline runs on a protected branch or tag, only runners marked protected can run the job, unless the job explicitly allows unprotected runners. If the branch is unprotected, an unprotected runner can run it. Mismatched protection leaves the job Pending.

   Where: Project settings > Protected branches, and the runner's protected flag in the Runners list.

   Expected check: runner protected setting matches the job's ref protection requirements.

4. Inspect concurrency, queue, and maintenance mode

   An active runner at its concurrency limit queues new jobs. Maintenance mode also prevents new job pickup. Check the runner's concurrent setting and whether maintenance mode is enabled.

   Where: runner configuration on the runner host, or runner details in GitLab UI.

   Expected check: runner has free concurrency slots and is not in maintenance mode.

5. Review recent registration, version, and upgrade changes

   A runner that lost registration after a GitLab or Runner upgrade may appear in the list but not receive jobs. Version mismatches between GitLab and Runner can also change pending behavior.

   Where: runner host and GitLab version page.

   Expected check: runner is registered and its version is compatible with the GitLab instance.

Fixes tied to findings

• Tag mismatch: Align the job's tags in .gitlab-ci.yml with an existing runner, or add the missing tags to the runner. Do not edit tags on shared runners without checking other projects that depend on them.
• Runner unavailable or wrong scope: Register a runner at the correct scope, unpause it, or bring it back online. Confirm the project has access to a group or instance runner.
• Protected-ref mismatch: Set the runner protected flag to match protected jobs, or run the pipeline from an unprotected branch. Changing a runner to protected affects every job that runner may take, not only the current pipeline.
• Capacity: Increase concurrent in the runner configuration or add more runners. Pending under concurrency limits is normal queuing, not a match failure.
• Stale registration or version: Re-register the runner or upgrade it to a version compatible with your GitLab instance.

Concrete example

This configuration shows a job that requires two tags and a runner that advertises both.

# .gitlab-ci.yml
deploy:
  stage: deploy
  tags:
    - production
    - linux
  script:
    - echo "deploying to production"

# /etc/gitlab-runner/config.toml on the runner host
[[runners]]
  name = "prod-runner-1"
  tag_list = ["production", "linux"]
  run_untagged = false
  locked = false
  concurrent = 10

If the runner's tag_list contains only production, the job stays Pending because GitLab requires all job tags to match runner tags. Add linux to the runner, or remove linux from the job if another runner can handle it.

Minimal diagnostic pipeline

Create a temporary pipeline with one untagged job on an unprotected branch:

# .gitlab-ci.yml
probe:
  script:
    - echo "pending probe"

If this job leaves Pending, reintroduce one constraint at a time—tags, then protected branch, then runner scope—to isolate which setting blocks the original job.

Commands and checks

Most checks are in the GitLab UI. On the runner host, use the following with caution.

Replace placeholder values such as <runner-id> or <project-id> with actual IDs from the GitLab UI before using any API call. Do not restart the runner service during active deployments unless you can tolerate interrupted jobs.

Escalation criteria

Escalate to instance administrators or support when:

• The job remains Pending after tag, scope, and protection checks pass.
• Several projects are affected at the same time.
• A runner reports active but never receives jobs.
• The symptom began after a GitLab or Runner upgrade.

Capture the following before escalating: job ID, pipeline ID, runner ID, job tags, ref, protected flag, and timestamps when the job entered Pending.

Limitations and verification

Exact tag matching, protected-runner semantics, runner scoping, concurrency limits, and pending-queue behavior vary across GitLab and Runner releases. Confirm behavior against your deployed version rather than assuming defaults. SaaS shared runners have different tag, capacity, and availability constraints than self-managed runners.

Practical check: after applying a fix, trigger a new pipeline and confirm the job leaves Pending within the expected time. Review the runner job log to verify the runner picked up the job. If another project's pipelines break after changing runner tags or protection, revert that specific runner change.