HDFS Erasure Coding for Cold Data in Hadoop 3.x
Reduce HDFS storage costs by applying Reed-Solomon erasure coding to cold archival directories while keeping hot workloads on three-way replication.
Loading…
ReadMeFeed / Community knowledge
#erasure-coding
Real questions. Useful conversations. Find the people who know your stack.
hadoopHDFS Erasure Coding for Cold Data: Cutting Storage Overhead Without Breaking Reads
Triple replication costs 200 percent overhead for cold HDFS data. Erasure coding in Hadoop 3.x drops that to roughly 50 percent, but adds read and reconstruction cost. Here is how to decide, configure, and verify.
Cutting HDFS Storage Costs with Erasure Coding: A Practical Guide
Cut HDFS storage costs with erasure coding: apply a 4+2 policy to a directory, write a 500 MB file, verify 1.5× overhead, and understand trade‑offs like recovery latency and write‑once restrictions.
Cutting HDFS Storage Costs: Moving from 3x Replication to Erasure Coding
Stop wasting 200% extra storage on cold data. Learn how to implement HDFS Erasure Coding to reduce overhead to 50% while maintaining fault tolerance.
Cutting HDFS Storage Costs with Erasure Coding: When 1.5x Beats 3x
HDFS Erasure Coding in Hadoop 3.x cuts storage overhead from 3x replication to ~1.5x for cold data. Here's how Reed-Solomon policies work, how to apply them, and the read-performance trade-off.
Cutting HDFS Storage Costs in Half with Erasure Coding
HDFS Erasure Coding in Hadoop 3.x cuts raw storage overhead from 200% to ~50% using Reed-Solomon RS-6-3. This blog walks through enabling EC on a cold-data directory, the write/read trade-offs, and operational guardrails for production clusters.
HDFS Erasure Coding: Trading CPU for Disk on Cold Data
HDFS erasure coding cuts storage overhead from 200% to roughly 50% for cold data, at the cost of CPU and read complexity. Here's how to enable it and verify the savings.
Reducing Hadoop Storage Overhead with HDFS Erasure Coding
Reduce HDFS storage overhead from 3x to 1.5x using Erasure Coding in Hadoop 3.x. Learn how to configure policies and verify EC block types for cold data workloads.
HDFS Erasure Coding: Trading CPU for Half Your Storage Bill
HDFS Erasure Coding in Hadoop 3.x cuts storage overhead from 3× to ~1.5× for cold data. Here's how to enable RS-6-3, verify it with fsck, and weigh the CPU and latency trade-offs.