Cutting HDFS Storage Costs: When to Swap 3x Replication for Erasure Coding
Stop paying the 300% storage tax. Learn how to use HDFS Erasure Coding to reduce storage overhead while maintaining data durability for cold datasets.
ReadMeFeed / Community knowledge
Real questions. Useful conversations. Find the people who know your stack.
Stop paying the 300% storage tax. Learn how to use HDFS Erasure Coding to reduce storage overhead while maintaining data durability for cold datasets.
Deploy a minimal HDFS cluster that uses rack‑aware replication to guarantee data durability and fault tolerance. The article covers required configuration, trust boundaries, operational checks, failure modes, and when to scale or add security features.
Establishing a repeatable data development environment requires strict synchronization between the HDFS directory hierarchy and the Hive Metastore metadata. While Hive-managed tables automate the lifecycle of underlying files, maintaining consistency across different environment-lifecycles presents challenges when DDL scripts are versioned independently of t