Ubuntu 14.10 下Hadoop 错误集
2015-08-05 20:07
351 查看
1 FATAL org.apache.hadoop.ha.ZKFailoverController: Unable to start failover controller. Parent znode does not exist.
这个错误导致启动不了DFSZKFailoverController,从而不能选举出Active Node,导致了Hadoop两个NameNode都是Standby,我是这样做的停掉Hadoop所有进程,然后重新格式化Zookeeper
hdfs zkfc -formatZK
2 紧接着上个问题,再重新格式化过zookeeper之后,发现yarn启动不了了
2015-08-05 19:00:33,718 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore: Exception while executing a ZK operation. org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode at org.apache.zookeeper.KeeperException.create(KeeperException.java:111) at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:949) at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:915) at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:937) at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:934) at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:1076) at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithRetries(ZKRMStateStore.java:1095) at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doMultiWithRetries(ZKRMStateStore.java:934) at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doMultiWithRetries(ZKRMStateStore.java:948) at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.removeRMDTMasterKeyState(ZKRMStateStore.java:844) at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.removeRMDTMasterKey(RMStateStore.java:733) at org.apache.hadoop.yarn.server.resourcemanager.security.RMDelegationTokenSecretManager.removeStoredMasterKey(RMDelegationTokenSecretManager.java:99) at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.removeExpiredKeys(AbstractDelegationTokenSecretManager.java:371) at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.rollMasterKey(AbstractDelegationTokenSecretManager.java:348) at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager$ExpiredTokenRemover.run(AbstractDelegationTokenSecretManager.java:646) at java.lang.Thread.run(Thread.java:745) 2015-08-05 19:00:33,718 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore: Maxed out ZK retries. Giving up! 2015-08-05 19:00:33,718 INFO org.apache.hadoop.yarn.server.resourcemanager.security.RMDelegationTokenSecretManager: removing master key with keyID 55 2015-08-05 19:00:33,738 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore: Exception while executing a ZK operation. org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode at org.apache.zookeeper.KeeperException.create(KeeperException.java:111) at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:949) at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:915) at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:937) at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:934) at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:1076) at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithRetries(ZKRMStateStore.java:1095) at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doMultiWithRetries(ZKRMStateStore.java:934) at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doMultiWithRetries(ZKRMStateStore.java:948) at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.removeRMDTMasterKeyState(ZKRMStateStore.java:844) at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.removeRMDTMasterKey(RMStateStore.java:733) at org.apache.hadoop.yarn.server.resourcemanager.security.RMDelegationTokenSecretManager.removeStoredMasterKey(RMDelegationTokenSecretManager.java:99) at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.removeExpiredKeys(AbstractDelegationTokenSecretManager.java:371) at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.rollMasterKey(AbstractDelegationTokenSecretManager.java:348) at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager$ExpiredTokenRemover.run(AbstractDelegationTokenSecretManager.java:646) at java.lang.Thread.run(Thread.java:745)
但是这个问题,估计还是因为zookeeper造成的,但是我没任何修改,从新启动yarn。。。竟然成功了
start-yarn.sh
3 NameNode启动不了
java.lang.IllegalArgumentException: Unable to construct journal, qjournal://spark-1421-0000:8485;spark-1421-0003:8485;spark-1421-0004:8485;spark-1421-0005:8485;spark-1421-0006:8485/hadoop-journal at org.apache.hadoop.hdfs.server.namenode.FSEditLog.createJournal(FSEditLog.java:1593) at org.apache.hadoop.hdfs.server.namenode.FSEditLog.initJournals(FSEditLog.java:276) at org.apache.hadoop.hdfs.server.namenode.FSEditLog.initSharedJournalsForRead(FSEditLog.java:254) at org.apache.hadoop.hdfs.server.namenode.FSImage.initEditLog(FSImage.java:776) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:621) at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:281) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1020) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:739) at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:536) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:595) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:762) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:746) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1438) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1504) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.hdfs.server.namenode.FSEditLog.createJournal(FSEditLog.java:1591) ... 13 more Caused by: java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.hdfs.qjournal.client.IPCLoggerChannel at org.apache.hadoop.hdfs.qjournal.client.IPCLoggerChannel.<init>(IPCLoggerChannel.java:146) at org.apache.hadoop.hdfs.qjournal.client.IPCLoggerChannel$1.createLogger(IPCLoggerChannel.java:156) at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.createLoggers(QuorumJournalManager.java:367) at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.createLoggers(QuorumJournalManager.java:149) at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.<init>(QuorumJournalManager.java:116) at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.<init>(QuorumJournalManager.java:105) ... 18 more
我这里的错误关键是:
tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.hdfs.qjournal.client.IPCLoggerChannel
后来排查,发现是因为:com.google.common.base.Stopwatch这个类造成的,因为这个类在guava.jar中,但是这个jar包在Hadoop 中是11.0.2,但是在java_home/jre/lib/ext中还有一个是18.*,那么造成了版本不一致,所以我的解决办法是:删掉java_home/jre/lib/ext中的guava.jar
相关文章推荐
- shell计算日期之间的天数
- linux命令2——进程相关
- Geoserver+Openlayers3加载瓦片地图
- Linux输入子系统(转)
- OpenGL坐标变换
- Openjudge6039 大师兄,师傅被妖怪抓走啦(模拟)
- tomcat Host及Context 配置
- centos hostname - 显示或设置主机名
- Openjudge6044 鸣人和佐助(bfs)
- CentOS用户切换
- 有关nginx Tornado tomcat apache<抄次网上>
- shell脚本学习笔记之正则表达式
- hadoop深入了解(三)
- UVa 714 Copying Books
- openstack社区文档贡献
- linux LIBRARY_PATH
- Spark历险记之编译和远程任务提交
- Tomcat日志设置
- [2015hdu多校联赛补题]hdu5348 MZL's endless loop
- CentOS 7 grub Linux 修改默认的启动操作系统