Tiered Storage Offload Driver Configuration for Cloud-Native Environments

Apache Pulsar supports tiered storage to move cold data from BookKeeper ledgers to external object stores. While local development environments often utilize a local file system implementation for this feature, production deployments typically transition to cloud providers like AWS S3 or Google Cloud Storage.

A primary challenge arises when moving from a local filesystem to a cloud-based managedLedgerOffloadDriver. The transition introduces dependencies on external IAM roles, VPC peering, and different consistency models that are not present in local testing. Specifically, the interaction between the offload threshold and the latency of the external storage provider can impact read performance for consumers accessing offloaded data.

What are the recommended configuration parameters to ensure the broker maintains stability when the external storage provider experiences high latency or temporary access denials? How does the system handle metadata consistency during the transition from BookKeeper to an eventually consistent cloud store?