Mastering Webflow CMS API Pagination: Efficient, Rate‑Limited Collection Fetching

Problem: Pulling a Large Collection from Webflow

When building a headless front‑end that relies on Webflow’s CMS, developers often need to fetch hundreds or thousands of items. The CMS API limits each request to 100 records, and the API key is rate‑limited. A naive loop that simply increments an offset can easily hit rate limits or miss items if the collection changes mid‑sync. This article shows how to use the pagination parameters correctly, respect rate limits, and keep the client code in sync with the Designer.

Understanding Pagination Parameters

The API exposes two query parameters:

• limit – maximum number of items per page (1‑100, default 100).
• offset – zero‑based index of the first item to return.

Given a collection with N items, the number of pages needed is ceil(N / limit). The X-Total-Count header, returned on every request, tells you N without an extra round‑trip.

Handling Rate Limits

Webflow applies a per‑API‑key bucket. Each response contains:

• X-RateLimit-Remaining – how many calls you can still make.
• X-RateLimit-Reset – epoch seconds when the bucket resets.

To avoid throttling, read X-RateLimit-Remaining after each request. If it drops to 0, pause until X-RateLimit-Reset or implement exponential back‑off. This keeps the sync resilient even under heavy traffic.

Practical Example – Fetching 250 Items

Below is a Node.js snippet that pulls a collection of 250 items, respecting limits and rate‑limit headers. Replace {API_KEY}, {COLLECTION_ID}, and {SITE_ID} with your values.

const fetch = require('node-fetch');
const API_KEY = 'your-api-key';
const COLLECTION_ID = 'your-collection-id';
const SITE_ID = 'your-site-id';
const LIMIT = 100;

async function fetchAllItems() {
  let offset = 0;
  let totalCount;
  const allItems = [];

  while (true) {
    const url = `https://api.webflow.com/collections/${COLLECTION_ID}/items?limit=${LIMIT}&offset=${offset}`;
    const res = await fetch(url, {
      headers: {
        Authorization: `Bearer ${API_KEY}`,
        'accept-version': '1.0.0',
      },
    });

    if (!res.ok) {
      throw new Error(`Request failed: ${res.status} ${res.statusText}`);
    }

    // Respect rate limits
    const remaining = res.headers.get('X-RateLimit-Remaining');
    const reset = res.headers.get('X-RateLimit-Reset');
    console.log(`Remaining: ${remaining}, reset at: ${new Date(reset * 1000)}`);
    if (remaining === '0') {
      const wait = (parseInt(reset, 10) - Math.floor(Date.now() / 1000)) * 1000;
      console.log(`Rate limit reached, sleeping for ${wait / 1000}s`);
      await new Promise(r => setTimeout(r, wait));
    }

    const data = await res.json();
    allItems.push(...data.items);

    if (totalCount === undefined) {
      totalCount = parseInt(res.headers.get('X-Total-Count'), 10);
      console.log(`Total items in collection: ${totalCount}`);
    }

    offset += LIMIT;
    if (offset >= totalCount) break;
  }

  console.log(`Fetched ${allItems.length} items`);
  return allItems;
}

fetchAllItems().catch(console.error);

Key points from the snippet:

• Always read X-Total-Count on the first request to know how many pages to fetch.
• Increment offset by limit each loop.
• Check X-RateLimit-Remaining after each call and back‑off if it hits zero.
• The payload mirrors the Designer schema; if you add a new field in the Designer, add a corresponding property in your data handling logic.

Trade‑off: Offset vs. Cursor

Webflow’s offset pagination is simple but fragile. If items are added or removed between requests, the offset may skip or duplicate records. Webflow does not provide an ID‑based cursor or a push API for changes, so:

• For static or rarely‑changing collections, offset pagination works fine.
• For live data, consider re‑syncing from scratch or storing the last fetched id and re‑running the loop if you detect missing items.
• Alternatively, fetch the entire collection in a single request when N <= 100 to avoid pagination complexity.

Actionable Takeaways

• Use limit=100 and calculate pages with X-Total-Count.
• Implement rate‑limit handling using X-RateLimit-Remaining and X-RateLimit-Reset.
• Keep your client code in sync with Designer schema changes; otherwise, deserialization will fail.
• Monitor collection changes; if you notice gaps, trigger a full re‑sync or switch to a safer strategy.

With these patterns, you can reliably pull large CMS collections into a headless application, stay within Webflow’s limits, and maintain data consistency.

Diagram

Component
limit (max 100)
offset (zero‑based)
X-Total-Count (total items)
X-RateLimit-Remaining / Reset (bucket)