How to Paginate Large Jira Issue Searches with the REST API

Why Pagination Matters

When a JQL query can match hundreds of thousands of issues, a single GET /rest/api/2/search call quickly runs into timeout limits, large payloads, or server‑side caps. The REST API is designed for incremental retrieval: you ask for a slice of the result set, then request the next slice until you have everything you need.

The Pagination Mechanism

The search endpoint accepts three key query parameters:

• jql – the Jira Query Language string that defines the issue set.
• startAt – a zero‑based index of the first issue to return.
• maxResults – the maximum number of issues to return in one response.

The server replies with a JSON payload that contains:

• startAt: the index used in the request.
• maxResults: the number actually returned (may be lower than requested).
• total: the total number of issues visible to the caller.
• issues: an array of issue objects.

To walk through the entire set, increment startAt by the number of issues returned until startAt >= total or the issues array is empty.

Concrete Example

Below is a minimal curl loop that fetches all issues for project ABC in batches of 100. Replace {BASE_URL} with your Jira instance and {AUTH_TOKEN} with a bearer token that has read access to issues.

#!/usr/bin/env bash
BASE_URL="https://your-domain.atlassian.net"
JQL="project=ABC"
START=0
PER_PAGE=100
while true; do
  RESPONSE=$(curl -s -H "Authorization: Bearer {AUTH_TOKEN}" \
    "${BASE_URL}/rest/api/2/search?jql=${JQL}&startAt=${START}&maxResults=${PER_PAGE}")
  # Extract fields with jq (assumes jq is installed)
  COUNT=$(echo "$RESPONSE" | jq -r '.issues | length')
  TOTAL=$(echo "$RESPONSE" | jq -r '.total')
  echo "Fetched $COUNT of $TOTAL (startAt=$START)"
  # Exit if no more issues
  [ "$COUNT" -eq 0 ] && break
  START=$((START + COUNT))
  # Optional: pause to respect rate limits
  sleep 0.2
done

Key points:

• The script reads the actual maxResults and total from each response, not the values you requested.
• It stops when the returned array is empty, guarding against instances where total is truncated.
• The sleep call mitigates the risk of HTTP 429 responses on very large data sets.

Server‑Side Limits & Common Mistakes

Hard‑coded maxResults Ceiling

Even if you request maxResults=1000, many Jira instances enforce a ceiling of 50 or 100. Always rely on the maxResults value returned in the JSON, not the request parameter.

Inaccurate total

The total field reflects only issues the authenticated user can view. If your account has restrictive permissions, total may be lower than the count shown in the UI for the same JQL. On very large projects, Jira may approximate total to avoid heavy queries.

Skipping the Loop Condition

Assuming total / maxResults gives the exact page count can miss the last page. Always loop until startAt >= total or the issues array is empty.

Rate Limiting

Rapid, consecutive calls can trigger a 429 response. If you hit this, back off and retry with exponential back‑off. Adding a short sleep between requests is a simple mitigation.

Verification & Best Practices

• After pagination, sum the issues.length from all responses and compare to the total reported on the last page.
• Cross‑check that number against the issue count shown in the Jira UI for the same JQL and user.
• Run the loop against a small test project first to ensure the logic works before scaling up.
• Inspect response headers for X-RateLimit-Remaining if your Jira instance exposes them.
• Keep maxResults modest (e.g., 50–200) to reduce payload size and avoid server‑side truncation.

By following this pattern, you avoid timeouts, respect server limits, and reliably retrieve all issues that match your query.