Hortonworks has been on a tear in the last year. They have made the most noise compared to other Hadoop distributions with new features and community support. With the release of HDP 2.3 today, they are even closer to making Hadoop a mainstream product that both large and small companies can easily adopt.
This post will review some of these new features and how they can help companies adopt this exciting new technology.
Hortonworks has combined the last few years of support cases into their own HDP cluster and have come up with many best practices and configurations to help companies run their Hadoop installations much smoother. This new feature adds new dialog boxes and easy to use sliders to help configure your Hadoop cluster with ease.
YARN Capacity Scheduler
The new Capacity Scheduling tool is a web-based tool with sliders to scale values which makes configuring this complex tool much easier. You can add new queues easier and see the hierarchy between queues in a graphical manner.
They are finally here. For years, Cloudera Manager had a great dashboard builder and you could customize it any way you like. Now, Ambari is getting closer to mainstream with custom metric widgets. You can create custom KPIs for your cluster helping you keep an eye on the things that need attention over other indicators.
More Ambari Views
The end of Hue is upon us if you are using HDP. 2.3 introduces more Ambari “Views". Views is a plug-in type architecture in Ambari which allows 3rd parties to create UI applications inside of Ambari. Eventually there will be a full marketplace with different “apps" that you can install into Ambari and assign those to user groups. For the 2.3 Sandbox from HDP, there are a few new Views which are Hive, Files, CapacityScheduler, Pig and Tez.
Data-at-Rest Encryption Improvements
Even more improvements to Apache Ranger with HDFS transparent encryption, management of Kafka, Solr and multi-tentant YARN queries. Apache Atlas is also included in this release which is a Data Governance incubator Apache project. We’ll dedicated a post just to this topic in the coming weeks.
Finally a company that is being proactive about their support and all of the hidden intelligence in that department. SmartSense was created by HDP support which much like any other software support company has a treasure trove of data on customer issues. They created their own HDP cluster and have been doing analytics on this data. The results are a set of common errors, best practices and performance standards based on years of supporting some of the largest Hadoop clusters in the world. It delivers ongoing recommendations on your cluster and offers a long-range cluster resource guide as well as helping you with capacity planning.