Hbase：PleaseHoldException: Master is initializing

问题：Hbase配置正确，但是一旦在HBase Shell中用create语句就出现问题“Master is initializing”

[root@node1 ~]# hbase shell
2017-03-18 17:55:42,489 WARN  [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/app/hbase-1.2.4/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/app/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 1.2.4, r67592f3d062743907f8c5ae00dbbe1ae4f69e5af, Tue Oct 25 18:10:20 CDT 2016

hbase(main):001:0> create 'user','info'

ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2293)
at org.apache.hadoop.hbase.master.HMaster.checkNamespaceManagerReady(HMaster.java:2298)
at org.apache.hadoop.hbase.master.HMaster.ensureNamespaceExists(HMaster.java:2507)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1499)
at org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:463)
at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:55682)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2180)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)

集群的时钟要保证基本的一致。稍有不一致是可以容忍的，但是很大的不一致会造成奇怪的行为。 运行 NTP 或者其他什么东西来同步你的时间.

在HBase中，如果你查询的时候或者是遇到奇怪的故障，可以检查一下系统时间是否正确!

检查一下集群节点的时间：

[root@node1 ~]# date
Sat Mar 18 18:02:20 CST 2017

[root@node3 ~]# date
Sat Mar 18 23:19:43 CST 2017

明显时间不同步！

那就时间同步吧

没有想到NTP时间同步也出现这么多问题：

出现问题：no server suitable for synchronization found

[root@node3 ~]# ntpdate -u node1
18 Mar 23:30:31 ntpdate[8967]: no server suitable for synchronization found

出现问题：Server dropped: no data

[root@node3 ~]# ntpdate -d node1
18 Mar 23:38:33 ntpdate[8997]: ntpdate 4.2.6p5@1.2349-o Mon Feb  6 07:22:46 UTC 2017 (1)
Looking for host node1 and service ntp
host found : node1
transmit(192.168.11.81)
transmit(192.168.11.81)
transmit(192.168.11.81)
transmit(192.168.11.81)
transmit(192.168.11.81)
192.168.11.81: Server dropped: no data
server 192.168.11.81, port 123
stratum 0, precision 0, leap 00, trust 000
refid [192.168.11.81], delay 0.00000, dispersion 64.00000
transmitted 4, in filter 4
reference time:    00000000.00000000  Mon, Jan  1 1900  8:05:43.000
originate timestamp: 00000000.00000000  Mon, Jan  1 1900  8:05:43.000
transmit timestamp:  dc77d2fc.2e36ddc8  Sat, Mar 18 2017 23:38:36.180
filter delay:  0.00000  0.00000  0.00000  0.00000
0.00000  0.00000  0.00000  0.00000
filter offset: 0.000000 0.000000 0.000000 0.000000
0.000000 0.000000 0.000000 0.000000
delay 0.00000, dispersion 64.00000
offset 0.000000

18 Mar 23:38:37 ntpdate[8997]: no server suitable for synchronization found

再次检查NTP server：node1

[root@node1 ~]# ntpstat
synchronised to local net at stratum 11
time correct to within 7948 ms
polling server every 64 s

在ntp server上使用命令watch ntpq -p

[root@node1 ~]# watch ntpq -p
Every 2.0s: ntpq -p                                                                                                                                                  Sat Mar 18 18:38:30 2017
remote           refid  st t when poll reach   delay   offset  jitter
==============================================================================
210.72.145.44   .INIT.     16 u    -   64    0    0.000    0.000   0.000
gus.buptnet.edu 10.3.9.9    4 u   33   64    7   30.580  5759803   2.404
59-124-196-83.H .INIT.     16 u    -   64    0    0.000    0.000   0.000
*LOCAL(0)        .LOCL.     10 l   33   64    7    0.000    0.000   0.000

注意：

LOCAL的这个就是与自身同步的ntp server。

reach这个值，在启动ntp server服务后，这个值就从0开始不断增加，当增加到17的时候，从0到17是5次的变更，每一次是poll的值的秒数，是64秒*5=320秒的时间。

如果之后从ntp客户端同步ntp server还失败的话，用ntpdate –d来查询详细错误信息，再做判断。

[root@node1 ~]# ntpdate –d
Exiting, name server cannot be used: Temporary failure in name resolution (-3)
18 Mar 18:40:24 ntpdate[4363]: name server cannot be used: Temporary failure in name resolution (-3)

解决方法：设置DNS服务器

[root@node1 ~]# echo "nameserver 114.114.114.114" > /etc/resolv.conf
[root@node1 ~]# cat /etc/resolv.conf
nameserver 114.114.114.114

重启网络

[root@node1 ~]# service network restart