hadoop2.2.0稳定版本发布了
2013-10-20 22:41
375 查看
期待已久的HADOOP2.2.0稳定版本终于在2013年10月15号发布了,由于hadoop1.0时代出现了诸多的问题,在2.0将解决大部分问题,并且这是必然趋势,相信2.0时代将会很快替代1.0时代。以下是从官方网站摘取的主要Release Notes,详细请看官方说明。
YARN - A general purpose resource management system for Hadoop to allow MapReduce and other other data processing frameworks and services
High Availability for HDFS
HDFS Federation
HDFS Snapshots
NFSv3 access to data in HDFS
Support for running Hadoop on Microsoft Windows
Binary Compatibility for MapReduce applications built on hadoop-1.x
Substantial amount of integration testing with rest of projects in the ecosystem
A couple of important points to note while upgrading to hadoop-2.2.0:
HDFS - The HDFS community decided to push the symlinks feature out to a future 2.3.0 release and is currently disabled.
YARN/MapReduce - Users need to change ShuffleHandler service name from mapreduce.shuffle to mapreduce_shuffle.
比较有趣的是,上面有一个比较大的改变,那就是可以支持运行在windows平台。并且我下载源码后编译成64bit,安装在centos上测试了下,速度明显提升不少,让我很兴奋,另外在yarn-site.xml配置文件中,yarn.nodemanager.aux-services属性的值由原来的mapreduce.shuffle改为mapreduce_shuffle需要注意的地方,不然运行自带wordcount例子报错,由于时间关系,写的比较随意,以后有时间会详细写些教程。
YARN - A general purpose resource management system for Hadoop to allow MapReduce and other other data processing frameworks and services
High Availability for HDFS
HDFS Federation
HDFS Snapshots
NFSv3 access to data in HDFS
Support for running Hadoop on Microsoft Windows
Binary Compatibility for MapReduce applications built on hadoop-1.x
Substantial amount of integration testing with rest of projects in the ecosystem
A couple of important points to note while upgrading to hadoop-2.2.0:
HDFS - The HDFS community decided to push the symlinks feature out to a future 2.3.0 release and is currently disabled.
YARN/MapReduce - Users need to change ShuffleHandler service name from mapreduce.shuffle to mapreduce_shuffle.
比较有趣的是,上面有一个比较大的改变,那就是可以支持运行在windows平台。并且我下载源码后编译成64bit,安装在centos上测试了下,速度明显提升不少,让我很兴奋,另外在yarn-site.xml配置文件中,yarn.nodemanager.aux-services属性的值由原来的mapreduce.shuffle改为mapreduce_shuffle需要注意的地方,不然运行自带wordcount例子报错,由于时间关系,写的比较随意,以后有时间会详细写些教程。
<property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> <description>shuffle service that needs to be set for Map Reduce to run </description> </property>
相关文章推荐
- Hadoop 2.0稳定版本2.2.0新特性剖析
- 【开源】开发者新闻聚合APP 2.0.3发布(第二个稳定版本)
- Hadoop 2.2.0版本HDFS的HA配置
- FineUI v3.3.2发布!目前最稳定版本,五年陈酿!
- Hadoop安全访问控制开源组件:Apache Sentry 1.4版本发布
- 购物兔发布稳定版本1.0.6.39,稳定程度大大提高
- Hadoop2.2.0版本多节点集群安装及测试
- Hadoop2.2.0版本多节点集群及eclipse开发环境搭建
- hadoop最新稳定版本使用建议
- Ruby 2.0.0 首个稳定版本(p0)发布
- Oushu HAWQ++ 2.2.0版本发布
- 搭建Hadoop 2.2.0版本HDFS的HA配置
- Apache Hadoop 2.2.0 (2013年10月发布通告)
- Hadoop2.2.0稳定版整合ZooKeeper(2)
- Hadoop2.2.0版本多节点集群安装及测试
- Hadoop2.2.0版本多节点集群安装及测试
- (大数据之hadoop)Hadoop2.2.0版本多节点集群及eclipse开发环境搭建
- Hadoop常用端口(以Hadoop2.2.0为例,不保证其他版本)
- Apache Beam发布第一个稳定版本
- Hadoop2.2.0版本多节点集群安装及测试