centos7下调试集群三台机器实现免密登陆--hadoop安装系列之一

一、实验目标

安装三台centos7虚拟机，组建hadoop实验集群，centos是从centos7官网下载的最新版本，默认系统安装，创建hadoop用户组，新建用户hadoop并加入hadoop组。

二、实验环境介绍

三台机器的网络主机配置如下：

192.168.10.166 master

192.168.10.167 slave01

192.168.10.168 slave02

hadoop官网下载链接：选择2.7.3稳定版
http://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.7.3/hadoop-2.7.3.tar.gz
javasdk需要在oralce的官方网站下载，目前是版本是jdk1.8
http://download.oracle.com/otn-pub/java/jdk/8u121-b13/e9e7ea248e2c4826b92b3f075a80e441/jdk-8u121-linux-x64.rpm?AuthParam=1491112154_2109c06de9b8b6eb3cd4e31e09df1780
操作系统版本如下：

[hadoop@master .ssh]$ uname -a

Linux master 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

[hadoop@master .ssh]$ java -version

java version "1.8.0_121"

Java(TM) SE Runtime Environment (build 1.8.0_121-b13)

Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

[hadoop@master .ssh]$ javac -version

javac 1.8.0_121

二、实验步骤

默认系统安装完后，分别进入master执行以下操作

1、生成密钥文件.
$ ssh-keygen 
然后一路回车.

[hadoop@master .ssh]$ ssh-keygen

Generating public/private rsa key pair.

Enter file in which to save the key (/home/hadoop/.ssh/id_rsa): 

Enter passphrase (empty for no passphrase): 

Enter same passphrase again: 

Your identification has been saved in /home/hadoop/.ssh/id_rsa.

Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.

The key fingerprint is:

ca:b9:86:98:4a:d8:8a:c5:c7:d8:f0:68:dd:c4:9f:f5 hadoop@master

The key's randomart image is:

+--[ RSA 2048]----+

|                 |

|                 |

|                 |

|     .           |

|  .   o S .      |

|.o O + + o .     |

|..Bo*.= o   E    |

|o+o.. ..         |

|+.   ..          |

+-----------------+
然后~/.ssh下会生成id_rsa.pub和id_rsa, 其中id_rsa文件起到唯一标识你的客户机的作用.

2、进入~/.ssh目录生成rsa默认配置文件authorized_keys 

[hadoop@master .ssh]$ cat id_rsa.pub >> authorized_keys 

--这里是一个坑，authorized_keys 文件的权限必须是600

[hadoop@master .ssh]$ chmod 600 authorized_keys 

[hadoop@master .ssh]$ ll

total 16

-rw-------. 1 hadoop hadoop  790 Apr  2 13:00 authorized_keys

-rw-------. 1 hadoop hadoop 1675 Apr  2 12:58 id_rsa

-rw-r--r--. 1 hadoop hadoop  395 Apr  2 12:58 id_rsa.pub

3、修改sshd配置文件(/etc/ssh/sshd_config).

--/etc/ssh/sshd_config文件是sshd服务的系统配置文件，修改这个文件需要root权限
找到以下内容，并去掉注释符”#“
=========================
　　RSAAuthentication yes
　　PubkeyAuthentication yes
　　AuthorizedKeysFile  .ssh/authorized_keys
=========================

默认配置前两项被注释掉，第一项RSAAuthentication表示允许进行RSA方式进行认证，第二项PubkeyAuthentication允许进行公钥认证，

第三项默认打开，但是如果不打开前两项，第三项不会生效，第三项配置的就是公钥认证文件的文件名及其所属目录，默认名字就是authorized_keys，你也可以修改，但是不建议这么做，保持约定俗成的约定对后期维护成本有不可低估的好处。

4、重启sshd服务，测试ssh本机是否可以免密登陆

[root@master
.ssh]# service sshd restart

--重启sshd服务需要root权限

Redirecting to /bin/systemctl restart  sshd.service

[hadoop@master
.ssh]$ ssh master

Last login: Sun Apr  2 12:10:41 2017 from master

--这一步只要不需要输入密码就成功了

5、~/.ssh目录如何自动生成

使用hadoop账户登陆服务器，然后使用ssh命令登陆一下任意一个其他主机，例如：

[root@slave02 .ssh]# ssh slave01

--在slave02使用hadoop账户只要执行一次ssh命令，系统就会自动在对应账户下生成一个隐形文件夹.ssh，用于保存ssh登陆的相关信息，一般不需要自己手工创建

The authenticity of host 'slave01 (192.168.10.167)' can't be established.

ECDSA key fingerprint is 1b:50:d1:5f:66:98:11:9f:38:ef:2c:2f:18:ea:d1:43.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'slave01,192.168.10.167' (ECDSA) to the list of known hosts.

root@slave01's password: 

6、拷贝master的公钥文件到集群的其他两台主机上；

完成这一步前一定要先在slave01及02上确保hadoop目录下存在.ssh目录，否则拷贝会提示失败。

[hadoop@master .ssh]$ scp authorized_keys hadoop@slave01:/home/hadoop/.ssh/

hadoop@slave01's password: 

authorized_keys                                                                                           100%  790     0.8KB/s   00:00    

[hadoop@master .ssh]$ scp authorized_keys hadoop@slave02:/home/hadoop/.ssh/

hadoop@slave02's password: 

authorized_keys                                                                                           100%  790     0.8KB/s   00:00    

7、登陆到slave01与slave02上修改各自/etc/ssh/sshd_config

具体修改步骤与在master上修改的步骤完全一致，参见第3步；

8、测试免密登陆是否成功

[hadoop@master .ssh]$ ssh slave01

Last login: Sun Apr  2 12:15:36 2017 from master

[hadoop@slave01 ~]$ exit

logout

Connection to slave01 closed.

[hadoop@master .ssh]$ ssh slave02

Last login: Sun Apr  2 12:17:16 2017 from master

[hadoop@slave02 ~]$ exit

logout

Connection to slave02 closed.

[hadoop@master .ssh]$ 

三、故障处理

1、无法实现免密登陆故障检查思路

  首先从master侧检查是否可以免密登陆自己，

   a）检查sshd的配置文件是否修改正确；

  b）sshd服务是否在修改文件重启成功；

  c）authorized_keys文件权限是否是600；

2、调试方法,使用ssh加上 -v参数进入调试模式观察调试信息寻找故障点

[hadoop@master .ssh]$ ssh -v slave01

OpenSSH_6.6.1, OpenSSL 1.0.1e-fips 11 Feb 2013

debug1: Reading configuration data /etc/ssh/ssh_config

debug1: /etc/ssh/ssh_config line 56: Applying options for *

debug1: Connecting to slave01 [192.168.10.167] port 22.

debug1: Connection established.

debug1: identity file /home/hadoop/.ssh/id_rsa type 1

debug1: identity file /home/hadoop/.ssh/id_rsa-cert type -1

debug1: identity file /home/hadoop/.ssh/id_dsa type -1

debug1: identity file /home/hadoop/.ssh/id_dsa-cert type -1

debug1: identity file /home/hadoop/.ssh/id_ecdsa type -1

debug1: identity file /home/hadoop/.ssh/id_ecdsa-cert type -1

debug1: identity file /home/hadoop/.ssh/id_ed25519 type -1

debug1: identity file /home/hadoop/.ssh/id_ed25519-cert type -1

debug1: Enabling compati
a645
bility mode for protocol 2.0

debug1: Local version string SSH-2.0-OpenSSH_6.6.1

debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1

debug1: match: OpenSSH_6.6.1 pat OpenSSH_6.6.1* compat 0x04000000

debug1: SSH2_MSG_KEXINIT sent

debug1: SSH2_MSG_KEXINIT received

debug1: kex: server->client aes128-ctr hmac-md5-etm@openssh.com none

debug1: kex: client->server aes128-ctr hmac-md5-etm@openssh.com none

debug1: kex: curve25519-sha256@libssh.org need=16 dh_need=16

debug1: kex: curve25519-sha256@libssh.org need=16 dh_need=16

debug1: sending SSH2_MSG_KEX_ECDH_INIT

debug1: expecting SSH2_MSG_KEX_ECDH_REPLY

debug1: Server host key: ECDSA 1b:50:d1:5f:66:98:11:9f:38:ef:2c:2f:18:ea:d1:43

debug1: Host 'slave01' is known and matches the ECDSA host key.
debug1: Found key in /home/hadoop/.ssh/known_hosts:1
--从本地known_hosts文件中发现曾经登陆过slave01，保存的有slave01的指纹信息

debug1: ssh_ecdsa_verify: signature correct

debug1: SSH2_MSG_NEWKEYS sent

debug1: expecting SSH2_MSG_NEWKEYS

debug1: SSH2_MSG_NEWKEYS received

debug1: SSH2_MSG_SERVICE_REQUEST sent

debug1: SSH2_MSG_SERVICE_ACCEPT received

debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password

debug1: Next authentication method: gssapi-keyex

debug1: No valid Key exchange context

debug1: Next authentication method: gssapi-with-mic

debug1: Unspecified GSS failure.  Minor code may provide more information

No Kerberos credentials available (default cache: KEYRING:persistent:1000)

debug1: Unspecified GSS failure.  Minor code may provide more information

No Kerberos credentials available (default cache: KEYRING:persistent:1000)

debug1: Next authentication method: publickey

debug1: Offering RSA public key: /home/hadoop/.ssh/id_rsa

--如果没有这两步检查配置文件是否修改正确并且是否重启过sshd服务；

debug1: Server accepts key: pkalg ssh-rsa blen 279

debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type RSA

debug1: Authentication succeeded (publickey).

--如果没有这两步检查文件authorized_keys是否存在并且权限是否是600
Authenticated to slave01 ([192.168.10.167]:22).

debug1: channel 0: new [client-session]

debug1: Requesting no-more-sessions@openssh.com

debug1: Entering interactive session.

debug1: Sending environment.

debug1: Sending env XMODIFIERS = @im=ibus

debug1: Sending env LANG = en_US.UTF-8

Last login: Sun Apr  2 13:18:27 2017 from master

四、总结

免密登陆是hadoop集群机器间，master主机对两台slave主机上的服务进行无交互自动调度所需要的操作系统约束，所以在部署hadoop集群前必须首先解决这个问题，

免密登陆的原理是把自己的rsa公钥文件送给需要免密登陆的主机.ssh目录下，让对方认得自己，比如本例中只是需要master可以免密登陆其它两台主机，所以只是把master的公钥文件送到了slave机器上，所以从slave主机上向master方向是不能免密登陆的，除非把slave主机的公钥文件送到master主机的.ssh目录下。