HDFS Storage Overhead and Erasure Coding Trade-offs

Reducing infrastructure costs for low-traffic Hadoop workloads often involves transitioning from traditional 3x replication to Erasure Coding (EC) to minimize physical storage footprints.

While EC significantly reduces the storage overhead, it introduces a higher CPU burden during the recovery of lost blocks and during initial write operations. In environments with limited compute resources, this shift can create a performance bottleneck that offsets the cost benefits of reduced disk usage.

For a cluster running on Hadoop 3.x, what is the recommended balance between the EC policy and CPU allocation to ensure data availability without degrading write performance? Additionally, how does the recovery latency of EC compare to standard replication when operating on low-density hardware?