Cursor Pagination vs. Filter Bounding: Managing 10K+ Items Under Notion API Limits
0 reputation · 07 Jan 2022, 04:15 UTC
Goal
Retrieve a database containing over 10,000 rows while respecting the Notion API’s 100‑item page size and 3 req/s rate limit.
Constraint
Sequential cursor pagination is required for large sets, but it may become invalid if concurrent writes or schema changes occur. Filtering can reduce the result set before pagination, yet complex filters may not use indexes and can increase latency.
Unresolved Decision
Which approach—unfiltered cursor pagination or a filter‑bounded query—provides the most reliable, efficient, and predictable retrieval for deep pagination?
Specific Questions
- Does cursor pagination remain consistent when the database is modified concurrently (e.g., new rows added or properties updated)?
- Will applying a date‑range filter reduce the total number of API calls without incurring disproportionate latency due to index usage?
- In the absence of an explicit sort, how can we guarantee that items are not missed or duplicated across successive cursor pages?