Exporting Large Notion Databases: Cursor‑Based Pagination and Server‑Side Filtering

Why Cursor‑Based Pagination Works for Notion

When you need to export or sync a Notion database that contains more than 100 pages, the API’s /v1/databases/{database_id}/query endpoint forces you to use cursor‑based pagination. The response includes a has_more flag and a next_cursor token that you pass back as start_cursor in the next request. This pattern guarantees you receive every page in the correct order without missing or duplicating data, even if the database is being edited concurrently.

Building the Query Request

All query requests are POST calls that must include the Authorization header, a Notion-Version header, and a JSON body. The body can specify page_size, filter, and sorts. The page_size range is 1–100; setting it to 100 reduces the number of round‑trips for large tables.

const query = {
  page_size: 100,
  filter: {
    property: 'Status',
    select: { equals: 'Done' }
  },
  sorts: [
    { property: 'Date', direction: 'descending' }
  ]
};

Iterating Through All Pages

Below is a minimal Node.js example that pulls all pages matching the filter, handling pagination until has_more is false. The code uses fetch (node‑fetch), but any HTTP client works.

async function fetchAll(dbId, token) {
  let cursor = undefined;
  const allPages = [];
  do {
    const body = { page_size: 100, filter: { property: 'Status', select: { equals: 'Done' } }, sorts: [{ property: 'Date', direction: 'descending' }] };
    if (cursor) body.start_cursor = cursor;
    const res = await fetch(`https://api.notion.com/v1/databases/${dbId}/query`, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${token}`,
        'Notion-Version': '2022-06-28',
        'Content-Type': 'application/json'
      },
      body: JSON.stringify(body)
    });

    if (!res.ok) {
      const err = await res.json();
      throw new Error(`API error ${res.status}: ${err.message}`);
    }

    const data = await res.json();
    allPages.push(...data.results);
    cursor = data.has_more ? data.next_cursor : undefined;
  } while (cursor);
  return allPages;
}

Server‑Side Filtering and Sorting

Filters and sorts are evaluated on Notion’s servers before pagination. This means you can reduce the payload size and avoid pulling pages you’ll discard later. The filter syntax mirrors the UI: a property name, a type (select, multi_select, text, etc.), and a comparison operator. Compound filters are built with and or or groups.

Filter Type	Example
Single select equals	{"property":"Status","select":{"equals":"Done"}}
Multi‑select contains	{"property":"Tags","multi_select":{"contains":"Bug"}}
Text contains	{"property":"Title","rich_text":{"contains":"API"}}

Rate Limiting and Back‑off

Notion enforces an average limit of roughly 3 requests per second per integration, with short bursts accepted. Exceeding the limit returns HTTP 429 Too Many Requests and a Retry-After header indicating seconds to wait. Production code should catch 429, sleep for Retry-After + a small jitter, then retry the same request.

if (res.status === 429) {
  const wait = parseInt(res.headers.get('Retry-After'), 10) + 200;
  await new Promise(r => setTimeout(r, wait));
  return fetchAll(dbId, token); // retry
}

Common Mistakes to Avoid

• Using GET instead of POST: the /query endpoint only accepts POST.
• Ignoring has_more: truncating results after the first page loses data.
• Missing the Notion-Version header: the API defaults to the latest version, which may change behavior.
• Not sharing the database with the integration: results in object_not_found errors even with the correct ID.
• Using wrong property types in filters: a select filter on a multi_select field throws a 400 validation error.
• Assuming offset‑based pagination: using a numeric start_cursor will fail; it must be the token returned by the API.

Practical Export Strategy for Production

For a nightly sync you can:

1. Set page_size to 100.
2. Use a last_edited_time filter to pull only pages modified since the last run.
3. Persist the last processed last_edited_time as a watermark.
4. Handle 429 back‑off and retry until the dataset is fully exported.
5. Log the number of pages processed and any errors for audit.

This approach avoids full rescans and keeps the sync fast even for databases with thousands of entries.

Verifying Your Export

After running the script, compare the length of allPages with the row count shown in the Notion UI. For large databases you can also spot‑check a few page IDs against the Notion web interface to confirm they were fetched correctly. If you hit a 429 during a test run, verify that the Retry-After header is respected and that the request count stabilizes below 3 per second.

Limitations

• Page objects can contain up to 2000 characters per rich‑text block; long text must be split when writing back to Notion.
• Rate limits are per integration, not per user, so multiple services using the same token share the quota.
• New API revisions may change the filter schema; always pin a Notion-Version and test against the current docs.