Cosmos DB Pagination: Why Continuation Tokens Beat OFFSET/LIMIT for Large Datasets

The problem: paging through millions of documents without burning RUs

You have a Cosmos DB container with 500,000 orders. A dashboard needs to show them 50 at a time. The intuitive approach—OFFSET 1000 LIMIT 50—works fine for page 1. By page 10,000, each request scans 500,000 documents before returning 50. The RU charge grows linearly with the offset. That’s not a bug; it’s how OFFSET/LIMIT works in Cosmos DB’s SQL API.

The alternative is server-side continuation tokens. They’re opaque strings the service returns when a query has more results. Pass the token back on the next call and the engine resumes exactly where it stopped—no re-scan, no offset math. The trade-off: tokens expire, have a size ceiling, and require a loop that handles throttling mid-iteration.

How continuation tokens actually work

When you issue a query (REST or SDK), the service executes it up to two limits: maxItemCount (page size you request) and the RU budget for that request. If either limit is hit before the full result set is drained, the response includes a continuation token (x-ms-continuation header in REST, ContinuationToken property in SDKs) and a flag indicating more results exist.

Crucially, a page can return fewer items than maxItemCount even when more data exists, because the RU budget for that request was exhausted. Client code must therefore loop on the continuation token, not on page size. In the .NET SDK v3, the pattern looks like this:

var iterator = container.GetItemQueryIterator<Order>(
    new QueryDefinition("SELECT * FROM c WHERE c.status = @status")
        .WithParameter("@status", "shipped"),
    requestOptions: new QueryRequestOptions { MaxItemCount = 50 }
);

while (iterator.HasMoreResults)
{
    FeedResponse<Order> page = await iterator.ReadNextAsync();
    foreach (var order in page) { /* process */ }
}

HasMoreResults is the reliable signal. Checking page.Count == 0 is wrong—a page can be empty while HasMoreResults remains true (rare, but possible with complex queries).

The OFFSET/LIMIT trap

OFFSET 50000 LIMIT 50 still forces the query engine to evaluate the predicate across the first 50,000 matching documents. You pay RUs for every document scanned, not just the 50 returned. On a 10 GB container with a non-selective filter, that’s thousands of RUs per page. Continuation tokens avoid this by persisting the scan position server-side.

There is a valid use case for OFFSET/LIMIT: ad-hoc exploration where you need to jump to page 42 without iterating 1 through 41. For any sequential access pattern—dashboards, exports, background jobs—continuation tokens are cheaper and scale linearly with result count, not offset depth.

Worked example: resilient paging with retry

Throttling (HTTP 429) can occur on any page request. The SDK surfaces this as CosmosException with StatusCode == 429. A production-grade iterator wraps ReadNextAsync in a retry policy that respects the RetryAfter header:

var policy = Policy
    .Handle<CosmosException>(ex => ex.StatusCode == HttpStatusCode.TooManyRequests)
    .WaitAndRetryAsync(
        retryCount: 5,
        sleepDurationProvider: attempt => TimeSpan.FromSeconds(Math.Pow(2, attempt)),
        onRetry: (ex, delay) => log.Warning("Throttled, retrying in {Delay}s", delay.TotalSeconds)
    );

while (iterator.HasMoreResults)
{
    FeedResponse<Order> page = await policy.ExecuteAsync(() => iterator.ReadNextAsync());
    foreach (var order in page) { /* process */ }
}

Run this against a container with 200+ orders and MaxItemCount = 2. You’ll see HasMoreResults stay true for ~100 iterations, then flip false on the final page. Log the ContinuationToken each loop—it grows as the query spans partitions.

Limitations you’ll hit in practice

• Token size ceiling (~16 KB). Cross-partition queries accumulate partition-range metadata in the token. A query scanning 50 partitions can exceed the limit, causing the request to fail with 400 Bad Request. Mitigation: narrow the partition key filter, or accept OFFSET/LIMIT for that specific query.
• Tokens are not durable cursors. They’re tied to the query’s execution state and the container’s snapshot at that moment. Schema changes, index rebuilds, or TTL deletions can invalidate them. Don’t persist tokens across deployments or long pauses.
• SDK version drift. Older .NET SDKs required EnableCrossPartitionQuery = true on FeedOptions; v3 enables it by default. Java and Python SDKs expose similar flags with different names. Verify the flag for your SDK version before assuming cross-partition works.

Verify it in your own container

1. Create a test container with a simple partition key (e.g., /tenantId).
2. Insert 200 documents with the same partition key.
3. Run a query with MaxItemCount = 5 and log ContinuationToken, HasMoreResults, and RequestCharge per page.
4. Confirm the sum of RequestCharge across pages matches a single non-paged query’s charge (within rounding).
5. Repeat with OFFSET 100 LIMIT 5 and compare the RU charge—expect it to be higher.

If the token size approaches 16 KB in step 3, you’ve found your cross-partition limit. That’s the signal to redesign the query or partition strategy, not to increase the token buffer.