Erasure Coding in HDFS

The folks at edureka! have an excellent post on some of the new features in Hadoop 3.
One of the features that really caught our eye is Erasure Encoding in HDFS. This is bringing RAID type architecture to HDFS to save a ton of storage. It’s like going from RAID 1 full mirroring to RAID 6. Some concerning issues would be stability, performance and rebuild times.

Here are some limitations so far in using this new technology from the Hortonworks documentation site:

If this new feature goes full GA and is stable, it could save our customers a lot of money in storage costs!

Check out the blog post here.

Here is a great in-depth blog post about Erasure Coding back in 2015 from Cloudera.