ReadMeFeed / Community knowledge
Real questions. Useful conversations. Find the people who know your stack.
Enable Kafka log compaction with cleanup.policy=compact, write keyed records and tombstones correctly, tune the dirty ratio and segment roll, and verify the cleaner kept only the latest value per key.
Kafka's idempotent producer makes retries safe by letting brokers discard duplicate batches. Here's how PID and sequence numbers work, what the guarantee doesn't cover, and how to verify it.
Enable idempotent producers everywhere by default. Reserve Kafka transactions for consume-transform-produce pipelines where cross-partition atomicity matters. Here's the configuration, trade-offs, and verification checklist.
Learn how to diagnose NotEnoughReplicas errors in Apache Kafka by analyzing ISR churn, adjusting replica lag timeouts, and verifying synchronization state.
Kafka’s Exactly‑Once Semantics (EOS) guarantees that each record is processed once, even across failures. This post walks through the problem, the EOS mechanism, how to enable it, a hands‑on Java example, trade‑offs, and how to verify your setup.
A single hot tenant can stall your entire Kafka consumer group. Adding partitions won't help — the fix is rethinking your record key. Composite keys (tenant_id:device_id) spread load while preserving per-entity ordering, but you lose global per-tenant order. Here's how to decide and verify.
Learn how Kafka’s idempotent and transactional producer APIs eliminate duplicate records caused by retries, and how to configure them safely for exactly‑once semantics in production pipelines.
Learn how to enable Kafka producer idempotence, verify the setup, and recover if it fails, ensuring each message is written exactly once to a topic partition.
Guide to selecting a pagination method for Kafka Streams state stores, comparing in‑memory, RocksDB, windowed and interactive query approaches with a concrete Java example.
apachekafka