lgK must be between 4 and 21 inclusive: ThetaSketch.Builder setLgK validation error

Pagination stability when using Apache DataSketches Theta Sketch for bounded query results

The goal is to use a Theta Sketch to estimate distinct element counts and derive page totals for paginating large datasets, while deciding whether to fix the lgK parameter globally for all queries or to adapt it per query based on observed cardinality.

Constraints include the library’s validation that lgK must be between 4 and 21 inclusive, throwing an IllegalArgumentException with the message “lgK must be between 4 and 21 inclusive” when the value is out of range, and the prohibition against merging sketches with different lgK values. Because the sketch’s estimate is probabilistic, page boundaries may shift between queries unless the same sketch configuration is reused.

• What are the trade‑offs of fixing lgK globally versus adapting it per query in terms of memory usage, accuracy, and pagination consistency?
• How does changing lgK affect the theoretical error bound and the resulting page‑count estimates?
• Is there a recommended approach to reuse a sketch instance across queries while respecting the library’s lgK validation and merge constraints?