Paginating LinkedIn Search Results Without Duplicates

Start with the offset, not the token

LinkedIn's Search API uses offset pagination: count sets the page size and start sets the zero-based offset. If your bulk extraction script treats start as durable across OAuth refreshes, you will skip records or fetch duplicates after a token rotation. The practical rule is to re-initialize start to zero whenever the access token is renewed, and to stop when a page returns fewer results than count or an empty array.

How the request parameters work

Every call needs an OAuth 2.0 Bearer token with the minimum scope for the data you need. For organization-level actions, that is often r_organization_social; for member profile search, a different scope may apply. The API accepts count (results per page, commonly capped at 50) and start (zero-based offset). A negative start or a count above the limit returns an error.

Worked extraction loop

Run this pattern in a local development environment or CI job that can store the access token securely. Replace LINKEDIN_ACCESS_TOKEN and SEARCH_QUERY with your own values. The script below is a structural example; it has not been executed against LinkedIn and produces no guaranteed output.

import time
import requests

BASE_URL = "https://api.linkedin.com/v2/search"
COUNT = 50
start = 0
headers = {
    "Authorization": "Bearer LINKEDIN_ACCESS_TOKEN",
    "X-Restli-Protocol-Version": "2.0.0",
}

while True:
    params = {
        "q": "companies",          # placeholder; adjust to your search type
        "count": COUNT,
        "start": start,
    }
    response = requests.get(BASE_URL, headers=headers, params=params, timeout=30)

    if response.status_code == 429:
        retry_after = int(response.headers.get("Retry-After", "60"))
        time.sleep(retry_after)
        continue

    response.raise_for_status()
    page = response.json()
    elements = page.get("elements", [])

    if not elements:
        break

    # process elements here
    start += len(elements)

    if len(elements) < COUNT:
        break

Required permissions: an active LinkedIn developer application, an approved OAuth flow, and a bearer token whose scope covers the endpoint. Expected checks: HTTP 200 on the first page, JSON containing an elements array, and termination when the array shrinks below COUNT or is empty. Risks: ignoring 429 can trigger application-level throttling; requesting broader scopes than needed can cause consent rejection.

Validate before you paginate

Call /v2/me with the same Bearer token and confirm HTTP 200 with the expected member fields. This verifies token validity and scope before you start a large extraction. While paginating, inspect X-RateLimit-Remaining and Retry-After to confirm the backoff behavior for your application.

Limits and common mistakes

• Offset does not survive token refresh. LinkedIn access tokens expire after 60 days. After re-authorization, reset start to zero; otherwise the next page may point to a position that no longer maps to the same result set.
• Result sets can be capped. LinkedIn may limit the total number of retrievable records for a search, so an empty array is not the only possible termination condition.
• Rate limits are per application. Roughly 100–200 requests per minute is a common envelope, but the exact threshold depends on your app and endpoint. Sleep only for the duration in Retry-After; do not poll faster.
• Negative offsets and oversized pages fail. Use zero-based start and keep count at or below the documented maximum, typically 50.

Practical check

After the loop finishes, compare the total records collected against the first page's total or metadata field if present. If the counts diverge, inspect whether a token refresh happened mid-extraction and whether start was reset. There is no rollback because this is a read-only operation.