Does omitting the 'last' cursor in Deta Base fetch operations produce latency spikes that mimic connection pool exhaustion?
0 reputation · 12 Jan 2023, 16:23 UTC
0 reputation · 12 Jan 2023, 16:23 UTC
Determine how Deta Base fetch pagination behavior without the 'last' cursor influences observed request latency and error patterns in deployed micros instances. Deta Space runs on managed Micros instances where the platform governs concurrent request handling, and the SDK abstracts underlying connection lifecycle, making it difficult to distinguish platform‑throttling from client‑side pagination errors. Item size limits and scan‑based query latency grow with base size, which can produce timeouts that resemble connection exhaustion.
The precise goal is to isolate whether intermittent resource‑exhaustion reports stem from cursor‑handling logic in application code or from the platform’s concurrency model, given that traditional connection‑pool diagnostics do not apply to Deta Base’s API. Because the 'last' cursor is a documented pagination feature, its omission or misuse provides a verifiable entry point for this investigation.
Does omitting the 'last' cursor in a fetch loop produce latency spikes that scale with base size? Can observed timeout patterns be differentiated from cursor‑handling failures without runtime instrumentation? Which concurrency thresholds in the Deta Space runtime interact most strongly with unbounded fetch loops?
33275 reputation · 12 Jan 2023, 18:57 UTC
Yes. Omitting the last cursor forces the SDK to restart each fetch from the beginning of the base, producing full scans whose latency grows superlinearly with base cardinality. On Deta Space, these long-running scans consume per-instance request quotas and wall-clock time, generating timeouts and 429/5xx responses that are indistinguishable from connection-pool exhaustion without additional instrumentation.
last the SDK issues a new scan from the first item on every call.fetch() call. Record whether last was supplied, the returned cursor, and wall-clock time.last cursor chaining, one without. Keep batch size identical.What is the typical base cardinality and average item size in the affected micros? If the base is under ~5k items and items are small (<1 KB), scan latency may be low enough that quota exhaustion is unlikely; in that case, investigate SDK retry logic or external network latency first. If the base exceeds 10k items or items approach the 1 MB limit, cursor omission is the primary suspect and the steps above will confirm it.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.