Geo‑replication design note for Apache Pulsar clusters
A concise architecture note on Pulsar geo‑replication: requirements, minimal primary‑standby design, trust boundaries, operational verification, failure modes, and triggers for redesign.
ReadMeFeed / Community knowledge
Real questions. Useful conversations. Find the people who know your stack.
A concise architecture note on Pulsar geo‑replication: requirements, minimal primary‑standby design, trust boundaries, operational verification, failure modes, and triggers for redesign.
Pulsar's tiered storage moves cold segments from BookKeeper to S3 or GCS automatically. Here's how offload triggers, how to configure and verify it, and the geo-replication and GC trade-offs to plan for.
Explains how Pulsar separates stateless brokers from stateful BookKeeper storage, outlines trust boundaries, operational checks, failure modes, and when the design might be altered.
Learn how to diagnose and fix stalled topic compaction in Apache Pulsar, including identifying I/O bottlenecks, misaligned retention policies, and ledger storage issues.
Apache Pulsar supports tiered storage to move cold data from BookKeeper ledgers to external object stores. While local development environments often utilize a local file system implementation for this feature, production deployments typically transition to cloud providers like AWS S3 or Google Cloud Storage. A primary challenge arises when moving from a loc
Goal and constraint Consider a namespace on a Pulsar 3.x cluster where recent messages must stay replayable if a deployment is rolled back. Only persistent topics backed by BookKeeper ledgers qualify, and bookie SSD capacity is limited, so retention cannot grow without bound. Tiered storage offload is available but adds retrieval latency for offloaded ledger