ReadMeFeed / Community knowledge
Real questions. Useful conversations. Find the people who know your stack.
Optimizing Storage for Low-Traffic Workloads Apache Pulsar supports tiered storage to move ledger segments from BookKeeper to external object stores like Amazon S3. For low-traffic workloads, reducing the offload threshold allows the system to minimize expensive disk usage by shifting data to cheaper object storage more aggressively. Constraints and Trade-of
We run a multi-tenant Apache Pulsar cluster (assume a recent 3.x broker; behavior may differ on older releases) and see lookup latency spikes that only appear when many clients issue topic lookups concurrently. Produce and consume latency stay flat, so the bottleneck appears to be broker-side lookup handling rather than BookKeeper or ZooKeeper throughput. Th
Message Timestamping and Regional Offsets Apache Pulsar (version 2.11.x) utilizes a UTC epoch millisecond value for eventTime . While this ensures a consistent global timeline, it places the burden of regional time zone conversion and date interpretation entirely on the producer and consumer applications. Consistency Constraints Currently, brokers do not int
Apache Pulsar supports tiered storage to move cold data from BookKeeper ledgers to external object stores. While local development environments often utilize a local file system implementation for this feature, production deployments typically transition to cloud providers like AWS S3 or Google Cloud Storage. A primary challenge arises when moving from a loc
Goal and constraint Consider a namespace on a Pulsar 3.x cluster where recent messages must stay replayable if a deployment is rolled back. Only persistent topics backed by BookKeeper ledgers qualify, and bookie SSD capacity is limited, so retention cannot grow without bound. Tiered storage offload is available but adds retrieval latency for offloaded ledger
pulsar