yarn.resourcemanager.ha.id设置
2016-11-18 21:59
337 查看
resourcemanager启动报错,其中一个启动成功,另一个启动报8088端口被成功启动的rm占用
标红的IP:172.16.145.114为成功启动的resourcemanager,上面的报错信息是启动失败的rm(172.16.145.115)报的错
错误显示114的8088端口已经在使用,跟我115有毛关系啊。
检查配置文件发现 两个rm中的yarn-site.xml中的 yarn.resourcemanager.ha.id设置成了同一个rm的主机名
解决办法:rm1的yarn-site的yarn.resourcemanager.ha.id设置成rm1
rm2的yarn-site的yarn.resourcemanager.ha.id设置成rm2
错误总结:一直以为两个resourcemanager配置文件是一样的,没有注意到这个属性。
对于这个属性两个rm的yarn-site设置不一样,而且slave中不需要配,其他属性暂时还发现,所有节点保持一致就行了
2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/lib/hadoop/lib/native 2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux 2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64 2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=2.6.32-504.el6.x86_64 2016-11-18 17:08:49,479 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=yarn 2016-11-18 17:08:49,479 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/var/lib/hadoop-yarn 2016-11-18 17:08:49,479 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/usr/lib/hadoop-yarn 2016-11-18 17:08:49,480 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=node01:2181,node02:2181,node03:2181 sessionTimeout=10000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@729b9707 2016-11-18 17:08:49,501 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server node02/172.16.145.112:2181. Will not attempt to authenticate using SASL (unknown error) 2016-11-18 17:08:49,507 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to node02/172.16.145.112:2181, initiating session 2016-11-18 17:08:49,518 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server node02/172.16.145.112:2181, sessionid = 0x2587635b7620006, negotiated timeout = 10000 2016-11-18 17:08:49,521 INFO org.apache.hadoop.ha.ActiveStandbyElector: Session connected. 2016-11-18 17:08:49,544 INFO org.apache.hadoop.ha.ActiveStandbyElector: Successfully created /yarn-leader-election/myclusterrm in ZK. 2016-11-18 17:08:49,549 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioning to standby state 2016-11-18 17:08:49,549 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to standby state 2016-11-18 17:08:49,642 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2016-11-18 17:08:49,648 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.resourcemanager is not defined 2016-11-18 17:08:49,666 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2016-11-18 17:08:49,677 INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster 2016-11-18 17:08:49,677 INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static 2016-11-18 17:08:49,677 INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs 2016-11-18 17:08:49,678 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster 2016-11-18 17:08:49,678 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2016-11-18 17:08:49,678 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2016-11-18 17:08:49,684 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /cluster/* 2016-11-18 17:08:49,684 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /ws/* 2016-11-18 17:08:49,701 INFO org.apache.hadoop.http.HttpServer2: HttpServer.start() threw a non Bind IOException java.net.BindException: Port in use: 172.16.145.114:8088 at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:890) at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:826) at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:273) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:903) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1014) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1145) Caused by: java.net.BindException: Cannot assign requested address at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:444) at sun.nio.ch.Net.bind(Net.java:436) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216) at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:885) ... 6 more
标红的IP:172.16.145.114为成功启动的resourcemanager,上面的报错信息是启动失败的rm(172.16.145.115)报的错
错误显示114的8088端口已经在使用,跟我115有毛关系啊。
检查配置文件发现 两个rm中的yarn-site.xml中的 yarn.resourcemanager.ha.id设置成了同一个rm的主机名
解决办法:rm1的yarn-site的yarn.resourcemanager.ha.id设置成rm1
rm2的yarn-site的yarn.resourcemanager.ha.id设置成rm2
<!--在node04上配rm1,node05上配rm2,slave节点不用配这个属性--> <property> <name>yarn.resourcemanager.ha.id</name> <value>rm1</value> </property>
错误总结:一直以为两个resourcemanager配置文件是一样的,没有注意到这个属性。
对于这个属性两个rm的yarn-site设置不一样,而且slave中不需要配,其他属性暂时还发现,所有节点保持一致就行了
相关文章推荐
- hadoop集群之YARN`s ResourceManager HA(三)
- Ambari—YARN配置ResourceManager HA(二)
- HDFS的Federation以及HA+YARN+ResourceManagerHA
- How-to: Enable yarn ResourceManager HA
- yarn 与 resource manager ha
- 通过tarball形式安装HBASE Cluster(CDH5.0.2)——配置分布式集群中的YARN ResourceManager 的HA
- Yarn启用HA后,删除定义的配置参数,如:yarn.resourcemanager.hostname
- Hadoop 管理工具HUE配置-Yarn Resource Manager HA配置
- Hadoop2 namenode HA + 联邦 + Resource Manager HA 实验
- hadoop集群搭建-Namenode HA和ResourceManager HA
- YARN ResourceManager调度器的分析
- Hadoop-2.3.0-cdh5.0.1完全分布式环境搭建(NameNode,ResourceManager HA)
- Hadoop-2.3.0-cdh5.0.1完全分布式环境搭建(NameNode,ResourceManager HA)
- YARN ResourceManager调度器的分析
- hadoop2中ResourceManager的HA
- 从Resource Manager中获取一个新的Application ID
- Enabling HDFS/YARN HA and Other CDH Components to Use HDFS HA with Cloudera Manager 5.2
- hadoop2.2.0 yarn-site.xml--Resource Manager Configs
- YARN ResourceManager调度器的分析
- YARN ResourceManager调度器的分析