Cutting HDFS Storage Overhead with Erasure Coding in Hadoop 3.x
HDFS erasure coding in Hadoop 3.x cuts storage overhead from 200% to ~50% for cold data. A worked RS-6-3 configuration, verification steps, and the limits that matter.
Loading…
ReadMeFeed / Community knowledge
#erasure-coding
Real questions. Useful conversations. Find the people who know your stack.
hadoopUsing HDFS Erasure Coding to Cut Storage Costs in Hadoop 3.x
Learn how to enable HDFS erasure coding in Hadoop 3.x to cut storage costs for cold data, with a step‑by‑step example and trade‑offs.
Cutting HDFS Storage Costs: When to Swap 3x Replication for Erasure Coding
Stop paying the 300% storage tax. Learn how to use HDFS Erasure Coding to reduce storage overhead while maintaining data durability for cold datasets.
Erasure Coding Policy Commitment in Hadoop 3.x for Low-Traffic Workloads
The objective is to evaluate whether erasure coding delivers net storage cost reduction for a low-traffic Hadoop 3.x workload without compromising read performance or increasing reconstruction latency. HDFS erasure coding, as documented in Hadoop 3.x, reduces storage overhead from the default three-way replication to approximately 1.5x for large, infrequentl
HDFS Storage Overhead and Erasure Coding Trade-offs
Reducing infrastructure costs for low-traffic Hadoop workloads often involves transitioning from traditional 3x replication to Erasure Coding (EC) to minimize physical storage footprints. While EC significantly reduces the storage overhead, it introduces a higher CPU burden during the recovery of lost blocks and during initial write operations. In environmen