ReadMeFeed / Community knowledge
Real questions. Useful conversations. Find the people who know your stack.
The constraint A producer must survive client-side retries (network timeouts, broker failovers) without writing duplicate records to a single-partition-per-key workload. Kafka documents two relevant mechanisms: the idempotent producer ( enable.idempotence=true with acks=all and retries>0 ), and full transactional writes via initTransactions() / commitTran
Context Kafka clients using the default ssl.endpoint.identification.algorithm=HTTPS verify the broker certificate hostname against the DNS name used at connection establishment. The verification occurs once during the initial handshake and is not repeated for the lifetime of that connection. Problem In environments with DNS-based failover or rolling broker u
Configuration boundary The min.insync.replicas setting determines the minimum in-sync replicas required for acks=all writes. Local single-broker clusters typically use default.replication.factor=1 and min.insync.replicas=1 , ensuring all writes succeed. Production environments often configure broker defaults with min.insync.replicas=2 or higher for durabilit
A small Kafka consumer application is being moved to a new deployment, and the goal is to keep message processing continuous while old and new instances briefly share the consumer group. Each membership change in that window triggers a rebalance, and the assignment protocol decides how much of the group stops consuming. Under an eager assignor such as RangeA
Design goal is to configure a Kafka topic for an integration test that uses log compaction together with time-based deletion, without access to production credentials. The topic should retain only the latest value per key under compaction, while messages without keys should be subject to delete retention. The cleanup.policy is intended to be compact,delete a
Producer Failure in High-Durability Configurations In a production Kafka cluster (version 3.x), a producer is configured with acks=all to ensure maximum data durability. While this configuration functions without issue in a local single-node environment, it triggers a NotEnoughReplicasException when deployed to a multi-broker production cluster. The cluster
apachekafka