hadoop配置总结

install  (hadoop-2.2.0.tar.gz + hbase-0.96.0-hadoop2-bin.tar.gz)

------------------------------------------------------------------------------

A:  ssh (root or other same user)

master:

1 ssh-keygen -t rsa

2 cp id_rsa.pub authorized_keys  

slave1 2 3: (same user as master_using )

1 ssh-keygen -t rsa

2 cp id_rsa.pub datanode1.pub          (datanode2 3 4...)

3 scp datanode1.pub master:/root/.ssh

master:

1 cat datanode1.pub >> authorized_keys

2 (cat datanode2.pub datanode3 4 5..... >> )

3 scp authorized_keys slave1:/root/.ssh/ (slave2 3 4....)

gedit /etc/ssh/sshd_config:

1 RSAAuthentication yes

          PubkeyAuthentication yes

          AuthorizedKeysFile .ssh/authorized_keys

2 PasswordAuthentication yes  (two ways of ssh_login)

3 GSSAPIAuthentication no     

 GSSAPIStrictAcceptorCheck no

gedit /etc/hosts

1 127.0.0.1 localhost 

2 192.168.0.110 slave1 

3 192.168.0.111 master

    ssh master; ssh slave1 
------------------------------------------------------------------------------

B hadoop2.2

 master:

1 hadoop-env.sh

2 core-site.xml

3 hdfs-site.xml 

4 yarn-site.xml

5 yarn-env.sh  

6 slaves

7 scp -r hadoop slave1:/usr/local/   (slave2 3 ....)

C hbase0.96(hbase-0.96.0-hadoop2-bin.tar.gz)

master:

1 hbase-env.sh

2 hbase-site.xml

3 regionservers

4 replace  /hbase/lib/hadoop-common-2.1.0-beta.jar

                   /hbase/lib/hadoop-hdfs-2.1.0-beta.jar

  /hbase/lib/hadoop-auth-2.1.0-beta.jar

 with         hadoop-common-2.2.0.jar

    hadoop-hdfs-2.2.0.jar

                   hadoop-auth-2.2.0.jar 

5 scp -r hbase slave1:/usr/local (slave2 3 ....)

D start hadoop 

1 bin/hdfs namenode -format

2 sbin/start-all.sh

3 jps (master)

4705 SecondaryNameNode
6722 Jps
4446 NameNode
4562 DataNode
4837 ResourceManager

4 master:50070 

E start hbase 

1 ./start-hbase

2 jps 

5537 Main
4705 SecondaryNameNode
5364 HMaster
4446 NameNode
5305 HQuorumPeer
6408 Jps
4562 DataNode
4837 ResourceManager

3 master:60010

4 ./hbase shell

list create put get 

----------------------------------------------------------------------------- 

reference :  

1 http://blog.csdn.net/java_kfc/article/details/17258657
2 http://blog.sina.com.cn/s/blog_72827fb10101a3dx.html 
Q:   Can't get connection to ZooKeeper: KeeperErrorCode = ConnectionLoss for /hba

A:  /etc/init.d/iptables stop 

3 setting error
-----------------------------------------------------------------------------

Question:

1 find the fit version -> hadoop 2.2 and hbase-0.96_hadoop2:  you will fail

2 /etc/init.d/iptables stop -> zoopkeeper stop connection failed...

3 del tmp and dfs/name ...  init 6  try again

4 start: ./hdfs namenode -format -> ./start-all.sh -> jps -> ./start-hbase.sh -> ./hbase shell

5 read log to find errors and ways 

6 different version different setting;  the best way to solve the problem is to read the document...