Choosing Cursor Pagination in Django REST Framework for Large, Dynamic Feeds

Problem: Page Numbers Fail on Dynamic Data

Imagine an API that returns a news feed of 10,000 posts. Clients request page 3, then new posts arrive, pushing existing items down. The next request for page 3 now returns a different set of posts, causing duplication or missing items on the client side. Page numbers are inherently fragile when the underlying queryset changes between requests.

For feeds that are frequently updated—social media timelines, order histories, or real‑time logs—stable ordering is essential. The solution in Django REST Framework (DRF) is cursor pagination, which uses an opaque token that encodes the ordering field and the last item retrieved. This token guarantees that subsequent requests continue from the exact position, regardless of inserts or deletes.

1. DRF’s Built‑in Pagination Classes

DRF ships with three pagination strategies:

Class	When to Use	Key Feature
PageNumberPagination	Simple, small lists	Page numbers, easy to understand
LimitOffsetPagination	Client‑controlled slicing	Offset + limit parameters
CursorPagination	Large, frequently changing datasets	Stable, deep pagination with opaque cursors

While page numbers are convenient for static lists, cursor pagination shines when the order must remain consistent across requests.

2. Why Cursor Pagination Wins for Dynamic Feeds

• Stable ordering – The cursor encodes the value of a specified ordering field (e.g., created_at) and a unique identifier. Even if new rows are inserted between requests, the cursor keeps the client from seeing duplicates.
• Efficient deep pagination – Instead of skipping rows with an offset, the database uses an index on the ordering field, making deep jumps fast.
• Scalable for large querysets – The cursor is a short string (<100 bytes) and does not expose internal IDs or offsets.

3. Configuring Cursor Pagination

Enable it globally in settings.py so every view uses it unless overridden:

# settings.py
REST_FRAMEWORK = {
    'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.CursorPagination',
    'PAGE_SIZE': 25,  # default page size
}

CursorPagination requires a stable ordering field. Add ordering = ['-created_at', 'id'] to your queryset or serializer. The second field ensures uniqueness when timestamps collide.

# views.py
from rest_framework import viewsets

class PostViewSet(viewsets.ModelViewSet):
    queryset = Post.objects.all().order_by('-created_at', 'id')
    serializer_class = PostSerializer
    pagination_class = None  # uses default from settings

When the client requests /api/posts/?cursor=eyJpZCI6MzY3fQ==, DRF decodes the base64 token, extracts the ordering values, and returns the next slice.

4. Fine‑tuning and Debugging

• Adjust page size per view – Override page_size in a viewset or use the ?page_size=50 query param if PAGE_SIZE_QUERY_PARAM is enabled.
• Check ordering consistency – Verify the ordering argument in your queryset matches the field used in the cursor. Mismatched ordering can skip items.
• Inspect the cursor – The cursor is base64‑encoded JSON. For debugging, decode it with echo <token> | base64 -d to see the ordering values.
• Validate links – The API response contains next and previous URLs. Ensure they are present and correctly formed by making successive requests with curl -i and inspecting the Link header or JSON body.

5. Trade‑offs & Limitations

Cursor pagination is powerful but not a silver bullet:

• Requires stable ordering – If your model’s ordering field changes frequently, the cursor can become invalid. Use immutable fields like primary keys or creation timestamps.
• Opaque tokens hide pagination state – Clients cannot jump to a specific page number; they must follow next links. This can be limiting for some UIs.
• Complex debugging – When items are inserted or deleted, tracing why a particular item is missing may require inspecting the cursor payload.
• Limited support for multi‑field ordering – While you can specify multiple fields, the cursor only guarantees uniqueness if the last field is unique (typically id).

6. Actionable Checklist

1. Define a stable ordering field (e.g., created_at + id) in the queryset.
2. Set DEFAULT_PAGINATION_CLASS to CursorPagination in REST_FRAMEWORK settings.
3. Verify the API returns next and previous links in the JSON response.
4. Test deep pagination: request the first page, record the cursor, request the next page repeatedly until the cursor is null.
5. Check that new items inserted between requests do not appear twice or get skipped by inspecting the ordering values in the cursor.
6. Document the cursor usage for front‑end developers: they should use the provided links rather than constructing cursors manually.

By following this guide, you’ll have a DRF API that scales gracefully to thousands of records, maintains consistent ordering, and delivers a smooth experience for clients that need to navigate deep into a constantly evolving dataset.