三、安装配置1.环境使用2台centos服务器，每台机器上部署3个实例，集群为三个主节点与三个从节点：192.168.5.144:6380192.168.5.144:6381192.168.5.144:6382192.168.5.8:6380192.168.5.8:6381192.168.5.8:63822.安装依赖由于通过redis-trib.rb工具构建Redis Cluster，需要rudy环境，执行如下命令安装：yum -y install zlib ruby rubygems安装ruby 的redis库：gem install redis3.安装redis 3.0.7wget
http://download.redis.io/releases/redis-3.0.7.tar.gz tar -zxvf redis-3.0.7.tar.gzmkdir rediscd redis-3.0.7make PREFIX=/home/slim/redismake PREFIX=/home/slim/redis install将集群工具复制到/home/slim/redis/bin下cp /home/slim/redis-3.0.7/src/redis-trib.rb ./bin/创建数据配置目录mkdir -p /home/slim/redis/{conf,data,logs}4.配置在2台机器上配置如下：cd /home/slim/rediscp /home/slim/redis-3.0.7/redis.conf ./conf/redis-6380.conf cp /home/slim/redis-3.0.7/redis.conf ./conf/redis-6381.conf cp /home/slim/redis-3.0.7/redis.conf ./conf/redis-6382.conf 修改配置：[plain]
view plain copy print?
#基本配置 daemonize yes pidfile /home/slim/redis/data/redis-6380.pid port 6380 bind 192.168.5.144 unixsocket /home/slim/redis/data/redis-6380.sock unixsocketperm 700 timeout 300 loglevel verbose logfile /home/slim/redis/logs/redis-6380.log databases 16 dbfilename dump-6380.rdb dir /home/slim/redis/data/ #aof持久化 appendonly yes appendfilename appendonly-6380.aof appendfsync everysec no-appendfsync-on-rewrite yes auto-aof-rewrite-percentage 80-100 auto-aof-rewrite-min-size 64mb lua-time-limit 5000 #集群配置 cluster-enabled yes cluster-config-file /home/slim/redis/data/nodes-6380.conf cluster-node-timeout 5000 每个实例配置类似，修改一下都应端口、IP地址、文件名称即可。Redis集群由多个运行在集群模式（cluster mode）下的Redis实例组成，实例的集群模式需要通过配置来开启，开启集群模式的实例将可以使用集群特有的功能和命令。要让集群正常运作至少需要三个主节点，不过在刚开始试用集群功能时， 强烈建议使用六个节点： 其中三个为主节点， 而其余三个则是各个主节点的从节点。cluster-enabled：开实例的集群模式cluster-conf-file：设定了保存节点配置文件的路径，默认值为nodes.conf。节点配置文件无须人为修改，它由Redis集群在启动时创建，并在有需要时自动进行更新。cluster-node-timeout：集群节点互连超时的阀值5.启动服务1）启动2台机器实例 ./bin/redis-server ./conf/redis-6380.conf ;tail -f logs/redis-6380.log ./bin/redis-server ./conf/redis-6381.conf ;tail -f logs/redis-6381.log ./bin/redis-server ./conf/redis-6382.conf ;tail -f logs/redis-6382.log这里启动的时候，可能会出现以下错误：-bash-3.2# ./bin/redis-server ./conf/redis-6380.conf ;tail -f logs/redis-6380.log./bin/redis-server: /lib64/libc.so.6: version `GLIBC_2.6' not found (required by ./bin/redis-server)tail: cannot open `logs/redis-6380.log' for reading: No such file or directorytail: no files remaining1.试图运行程序，提示"libc.so.6: version `GLIBC_2.6' not found",原因是系统的glibc版本太低，软件编译时使用了较高版本的glibc引起的：
[work@zc-mc-face01 src]$ ./redis-server
./redis-server: /lib64/libc.so.6: version `GLIBC_2.6' not found (required by ./redis-server)
2.查看系统glibc支持的版本：
strings /lib64/libc.so.6 | grep GLIBC_
GLIBC_2.2.5
GLIBC_2.2.6
GLIBC_2.3
GLIBC_2.3.2
GLIBC_2.3.3
GLIBC_2.3.4
GLIBC_2.4
GLIBC_2.5
GLIBC_2.6
GLIBC_2.7
GLIBC_2.8
GLIBC_2.9
GLIBC_2.10
GLIBC_2.11
GLIBC_2.12
GLIBC_PRIVATE
rpm -qa | grep glibc
glibc-common-2.12-1.80.el6_3.6.x86_64
glibc-2.12-1.80.el6_3.6.x86_64
glibc-headers-2.12-1.80.el6_3.6.x86_64
glibc-devel-2.12-1.80.el6_3.6.x86_64
3.可以看到当前系统最高只支持2.2版本，所以需要安装新版本来解决这个问题： 编译安装步骤如下： a. 到http://www.gnu.org/software/libc/下载最新版本，我这里下载了glibc-2.14.tar.gz 这个版本，解压到任意目录准备编译
wget http://ftp.gnu.org/gnu/glibc/glibc-2.6.tar.gz[/code] b.这里解压到～/glibc-2.6/ cd /home/work/glibc-2.6/
ls
BUGS ChangeLog.2 NAMESPACE bits elf libidn po string
CANCEL-FCT-WAIVE ChangeLog.3 NEWS build extra-lib.mk libio posix sunrpc
CANCEL-FILE-WAIVE ChangeLog.4 NOTES catgets extra-modules.mk locale pwd sysdeps
CONFORMANCE ChangeLog.5 PROJECTS conf gmon localedata resolv sysvipc
COPYING ChangeLog.6 README config.h.in gnulib login resource termios
COPYING.LIB ChangeLog.7 README.libm config.make.in grp mach rt test-skeleton.c
ChangeLog ChangeLog.8 Rules configure gshadow malloc scripts time
ChangeLog.1 ChangeLog.9 Versions.def configure.in hesiod manual setjmp timezone
ChangeLog.10 FAQ WUR-REPORT conform hurd math shadow tls.make.c
ChangeLog.11 FAQ.in abi-tags cppflags-iterator.mk iconv misc shlib-versions version.h
ChangeLog.12 INSTALL abilist crypt iconvdata nis signal wcsmbs
ChangeLog.13 LICENSES aclocal.m4 csu include nptl socket wctype
ChangeLog.14 Makeconfig aout ctype inet nptl_db soft-fp
ChangeLog.15 Makefile argp debug intl nscd stdio-common
ChangeLog.16 Makefile.in assert dirent io nss stdlib
ChangeLog.17 Makerules autom4te.cache dlfcn libc-abis o-iterator.mk streams
c.在glibc源码目录建立构建目录，并cd进入构建目录[work@zc-mc-face01 glibc-2.14]# mkdir build
[work@zc-mc-face01 glibc-2.14]# cd build
d.运行configure配置，make && sudo make install1. [work@zc-mc-face01 build]# ../configure --prefix=/opt/glibc-2.14
接下来继续运行2. [work@zc-mc-face01 build]# make -j4
3. [work@zc-mc-face01 build]# sudomakeinstall
4.临时修改环境变量[work@zc-mc-face01 build]# export LD_LIBRARY_PATH=/opt/glibc-2.14/lib:$LD_LIBRARY_PATH
5.接下来就可以正常启动redis了[work@zc-mc-face01 build]# cd ~/app/redis-2.8.13/src
[work@zc-mc-face01 src]# ./redis-server
启动日志中打印：“8591:M 11 Apr 22:20:46.134 * No cluster configuration found, I'm 83fc65283bbbb71b4c089337df05594d67f4cab6”每个节点都使用 ID 而不是 IP 或者端口号来记录其他节点， 因为 IP 地址和端口号都可能会改变， 而这个独一无二的标识符（identifier）则会在节点的整个生命周期中一直保持不变。查看启动进程：[plain] view plain copy print?ps -ef | grep redis slim 8525 1 0 22:16 ? 00:00:01 ./bin/redis-server 192.168.36.54:6380 [cluster] slim 8544 1 0 22:17 ? 00:00:00 ./bin/redis-server 192.168.36.54:6381 [cluster] slim 8591 1 0 22:20 ? 00:00:00 ./bin/redis-server 192.168.36.54:6382 [cluster] 2）集群配置在每台机器上启动的redis服务都是相互独立，下面我们就使用redis-trib.rb工具构建Redis Cluster。./bin/redis-trib.rb create --replicas 1 192.168.36.54:6380 192.168.36.54:6381192.168.36.54:6382 192.168.36.189:6380 192.168.36.189:6381 192.168.36.189:6382命令的意义如下：给定 redis-trib.rb 程序的命令是 create ， 这表示我们希望创建一个新的集群。选项 --replicas 1 表示我们希望为集群中的每个主节点创建一个从节点。之后跟着的其他参数则是实例的地址列表， 我们希望程序使用这些地址所指示的实例来创建新集群。简单来说， 以上命令的意思就是让 redis-trib 程序创建一个包含三个主节点和三个从节点的集群。启动日志：[plain] view plain copy print?>>> Creating cluster Connecting to node 192.168.36.54:6380: OK Connecting to node 192.168.36.54:6381: OK Connecting to node 192.168.36.54:6382: OK Connecting to node 192.168.36.189:6380: OK Connecting to node 192.168.36.189:6381: OK Connecting to node 192.168.36.189:6382: OK >>> Performing hash slots allocation on 6 nodes... Using 3 masters: 192.168.36.54:6380 192.168.36.189:6380 192.168.36.54:6381 Adding replica 192.168.36.189:6381 to 192.168.36.54:6380 Adding replica 192.168.36.54:6382 to 192.168.36.189:6380 Adding replica 192.168.36.189:6382 to 192.168.36.54:6381 M: f6285c8a7506b224840d7b26b2b5d1671320c21f 192.168.36.54:6380 slots:0-5460 (5461 slots) master M: 26ce71d626175f88e0416e3f45b2bfb29304c7b3 192.168.36.54:6381 slots:10923-16383 (5461 slots) master S: 83fc65283bbbb71b4c089337df05594d67f4cab6 192.168.36.54:6382 replicates b1a15a3cd14ea65671a7134850e17b8919a17da5 M: b1a15a3cd14ea65671a7134850e17b8919a17da5 192.168.36.189:6380 slots:5461-10922 (5462 slots) master S: 1080e423a55a2c24dae649dac03ffa09ed26d3e8 192.168.36.189:6381 replicates f6285c8a7506b224840d7b26b2b5d1671320c21f S: de4302f43ff89843675446396552fd19f741246a 192.168.36.189:6382 replicates 26ce71d626175f88e0416e3f45b2bfb29304c7b3 Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join... >>> Performing Cluster Check (using node 192.168.36.54:6380) M: f6285c8a7506b224840d7b26b2b5d1671320c21f 192.168.36.54:6380 slots:0-5460 (5461 slots) master M: 26ce71d626175f88e0416e3f45b2bfb29304c7b3 192.168.36.54:6381 slots:10923-16383 (5461 slots) master M: 83fc65283bbbb71b4c089337df05594d67f4cab6 192.168.36.54:6382 slots: (0 slots) master replicates b1a15a3cd14ea65671a7134850e17b8919a17da5 M: b1a15a3cd14ea65671a7134850e17b8919a17da5 192.168.36.189:6380 slots:5461-10922 (5462 slots) master M: 1080e423a55a2c24dae649dac03ffa09ed26d3e8 192.168.36.189:6381 slots: (0 slots) master replicates f6285c8a7506b224840d7b26b2b5d1671320c21f M: de4302f43ff89843675446396552fd19f741246a 192.168.36.189:6382 slots: (0 slots) master replicates 26ce71d626175f88e0416e3f45b2bfb29304c7b3 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. 从启动日志可以看出有6个集群实例，主节点为：192.168.36.54:6380192.168.36.189:6380192.168.36.54:6381每个主节点对应一个从节点Adding replica 192.168.36.189:6381 to 192.168.36.54:6380Adding replica 192.168.36.54:6382 to 192.168.36.189:6380Adding replica 192.168.36.189:6382 to 192.168.36.54:6381slots划分：192.168.36.54:6380 0-5460192.168.36.189:6380 5461-10922192.168.36.54:6381 10923-16383一共16384 slots注：redis-trib.rb使用参数[plain] view plain copy print?Usage: redis-trib <command> <options> <arguments ...> fix host:port call host:port command arg arg .. arg check host:port import host:port --from <arg> set-timeout host:port milliseconds add-node new_host:new_port existing_host:existing_port --master-id <arg> --slave reshard host:port --to <arg> --from <arg> --slots <arg> --yes create host1:port1 ... hostN:portN --replicas <arg> help (show this help) del-node host:port node_id For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster. 参数说明：call：执行redis命令create：创建一个新的集群。host1:port1 ... hostN:portN指定了用于构建Redis Cluster的所有redis实例，节点角色由顺序决定,先master之后是slave。而--replicas 则指定了为Redis Cluster中的每个Master节点配备几个Slave节点。add-node 将一个节点添加到集群里面， 第一个是新节点ip:port, 第二个是任意一个已存在节点ip:port，--master-id reshard：重新分片check：查看集群信息del-node：移除一个节点对于客户端redis-cli要访问集群，在启动的时候需要添加一个-c参数，如查看集群所有节点：[plain] view plain copy print?./bin/redis-cli -c -h 192.168.36.189 -p 6380 cluster nodes 1080e423a55a2c24dae649dac03ffa09ed26d3e8 192.168.36.189:6381 slave f6285c8a7506b224840d7b26b2b5d1671320c21f 0 1428817516066 5 connected b1a15a3cd14ea65671a7134850e17b8919a17da5 192.168.36.189:6380 myself,master - 0 0 4 connected 5461-10922 83fc65283bbbb71b4c089337df05594d67f4cab6 192.168.36.54:6382 slave b1a15a3cd14ea65671a7134850e17b8919a17da5 0 1428817515047 4 connected 26ce71d626175f88e0416e3f45b2bfb29304c7b3 192.168.36.54:6381 master - 0 1428817514946 2 connected 10923-16383 f6285c8a7506b224840d7b26b2b5d1671320c21f 192.168.36.54:6380 master - 0 1428817514844 1 connected 0-5460 de4302f43ff89843675446396552fd19f741246a 192.168.36.189:6382 slave 26ce71d626175f88e0416e3f45b2bfb29304c7b3 0 1428817516578 6 connected 参考文章：1.Redis 官方集群教程2.Redis集群原理与使用3.Redis3.0.0集群试练4.Redis3.0.0集群示例3down voteacceptedFor Ruby 1.8.6 I had to add another repository.Create /etc/yum.repos.d/ruby.repo and add the following:[ruby]name=rubybaseurl=http://repo.premiumhelp.eu/ruby/gpgcheck=0enabled=0Thenyum --enablerepo=ruby install rubyFrom www.centos.org forums.Don't bother installing rubygems this way, it's obsolete.Installing RubyGems Using apt-get on Ubuntu
sudo apt-get install rubygems
Installing RubyGems Using yum
sudo yum install rubygems
Manual Installation Method
wget http://production.cf.rubygems.org/rubygems/rubygems-1.5.0.tgz
tar xvf rubygems-1.5.0.tgz
cd rubygems-1.5.0
sudo ruby setup.rb
1执行上面的命令的时候会报错，因为是执行的ruby的脚本，需要ruby的环境错误内容：/usr/bin/env: ruby: No such file or directory所以需要安装ruby的环境，这里推荐使用yum install ruby安装
我的死活用yum 安装不成功，指定了rpm源还是不行，老是提示个别文件下载超时，没法办只能下载源文件编译。由于我的系统是centos5.4 自带的ruby是1.8.5，是不支持rubygems的。所以下载并解压Ruby 1.8.71.wget
http://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.7-p334.tar.gz 2.tar -xzvf ruby-1.8.7-p334.tar.gz3../configure -prefix=/usr/local -with-shared -without-debug4.make5.make install6.2然后再执行第6步的创建集群命令，还会报错，提示缺少rubygems组件，使用yum安装错误内容：./redis-trib.rb:24:in `require': no such file to load — rubygems (LoadError)from ./redis-trib.rb:24
如果yum 安装不成功，那么还是下载源码安装1.wget
http://production.cf.rubygems.org/rubygems/rubygems-1.6.2.tgz //最好是下载到/usr/local/src/目录中2.tar -xzvf rubygems-1.6.2.tgz3.cd rubygems-1.6.24.ruby setup.rb6.3再次执行第6步的命令，还会报错，提示不能加载redis，是因为缺少redis和ruby的接口，使用gem 安装错误内容：/usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require': no such file to load — redis (LoadError)from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require’from ./redis-trib.rb:25
6.4 再次执行第6步的命令，正常执行输入yes，然后配置完成。drwxr-xr-x 2 root root 4096 Jun 1 03:27 logs-bash-3.2# ./bin/redis-server ./conf/redis-6380.conf ;tail -f logs/redis-6380.log./bin/redis-server: /lib64/libc.so.6: version `GLIBC_2.6' not found (required by ./bin/redis-server)tail: cannot open `logs/redis-6380.log' for reading: No such file or directorytail: no files remaining-bash-3.2# wget
http://ftp.gnu.org/gnu/glibc/glibc-libidn-2.6.tar.gz--2016-07-18 17:12:29--
http://ftp.gnu.org/gnu/glibc/glibc-libidn-2.6.tar.gzResolving ftp.gnu.org... 208.118.235.20, 2001:4830:134:3::bConnecting to ftp.gnu.org|208.118.235.20|:80... connected.HTTP request sent, awaiting response... 200 OKLength: 127719 (125K) [application/x-gzip]Saving to: `glibc-libidn-2.6.tar.gz'100%[==========================================================================================>] 127,719 64.3K/s in 1.9s 2016-07-18 17:12:32 (64.3 KB/s) - `glibc-libidn-2.6.tar.gz' saved [127719/127719]以下在系统CentOS 6.3 x86_64上操作1.试图运行程序，提示"libc.so.6: version `GLIBC_2.14' not found",原因是系统的glibc版本太低，软件编译时使用了较高版本的glibc引起的：
[work@zc-mc-face01 src]$ ./redis-server
./redis-server: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by ./redis-server)
2.查看系统glibc支持的版本：
[work@zc-mc-face01 src]$strings /lib64/libc.so.6 | grep GLIBC_
GLIBC_2.2.5
GLIBC_2.2.6
GLIBC_2.3
GLIBC_2.3.2
GLIBC_2.3.3
GLIBC_2.3.4
GLIBC_2.4
GLIBC_2.5
GLIBC_2.6
GLIBC_2.7
GLIBC_2.8
GLIBC_2.9
GLIBC_2.10
GLIBC_2.11
GLIBC_2.12
GLIBC_PRIVATE
[work@zc-mc-face01 src]$ rpm -qa | grep glibc
glibc-common-2.12-1.80.el6_3.6.x86_64
glibc-2.12-1.80.el6_3.6.x86_64
glibc-headers-2.12-1.80.el6_3.6.x86_64
glibc-devel-2.12-1.80.el6_3.6.x86_64
3.可以看到当前系统最高只支持2.12版本，所以需要安装新版本来解决这个问题： 编译安装步骤如下： a. 到http://www.gnu.org/software/libc/下载最新版本，我这里下载了glibc-2.14.tar.gz 这个版本，解压到任意目录准备编译
wget http://ftp.gnu.org/gnu/glibc/glibc-2.14.tar.gz[/code] b.这里解压到～/glibc-2.14/ [work@zc-mc-face01 src]# cd /home/work/glibc-2.14/
[work@zc-mc-face01 glibc-2.14]# ls
BUGS ChangeLog.2 NAMESPACE bits elf libidn po string
CANCEL-FCT-WAIVE ChangeLog.3 NEWS build extra-lib.mk libio posix sunrpc
CANCEL-FILE-WAIVE ChangeLog.4 NOTES catgets extra-modules.mk locale pwd sysdeps
CONFORMANCE ChangeLog.5 PROJECTS conf gmon localedata resolv sysvipc
COPYING ChangeLog.6 README config.h.in gnulib login resource termios
COPYING.LIB ChangeLog.7 README.libm config.make.in grp mach rt test-skeleton.c
ChangeLog ChangeLog.8 Rules configure gshadow malloc scripts time
ChangeLog.1 ChangeLog.9 Versions.def configure.in hesiod manual setjmp timezone
ChangeLog.10 FAQ WUR-REPORT conform hurd math shadow tls.make.c
ChangeLog.11 FAQ.in abi-tags cppflags-iterator.mk iconv misc shlib-versions version.h
ChangeLog.12 INSTALL abilist crypt iconvdata nis signal wcsmbs
ChangeLog.13 LICENSES aclocal.m4 csu include nptl socket wctype
ChangeLog.14 Makeconfig aout ctype inet nptl_db soft-fp
ChangeLog.15 Makefile argp debug intl nscd stdio-common
ChangeLog.16 Makefile.in assert dirent io nss stdlib
ChangeLog.17 Makerules autom4te.cache dlfcn libc-abis o-iterator.mk streams
c.在glibc源码目录建立构建目录，并cd进入构建目录[work@zc-mc-face01 glibc-2.14]# mkdir build
[work@zc-mc-face01 glibc-2.14]# cd build
d.运行configure配置，make && sudo make install1. [work@zc-mc-face01 build]# ../configure --prefix=/opt/glibc-2.14
这一步如果config失败，显示错误如下：checking whether ranlib is necessary... no
checking LD_LIBRARY_PATH variable... contains current directory
configure: error:
*** LD_LIBRARY_PATH shouldn't contain the current directory when
*** building glibc. Please change the environment variable
*** and run configure again.
此时首先查看LD_LIBRARY_PATH：echo $LD_LIBRARY_PATH
/opt/soft/jdk/jre/lib/amd64/server:
而实际上这个路径的设置是有限制的：# Test if LD_LIBRARY_PATH contains the notation for the current directory
# since this would lead to problems installing/building glibc.
# LD_LIBRARY_PATH contains the current directory if one of the following
# is true:
# - one of the terminals (":" and ";") is the first or last sign
# - two terminals occur directly after each other
# - the path contains an element with a dot in it 所以这里我们只需要去掉 LD_LIBRARY_PATH 最后的那个路径分隔符':'即可：export LD_LIBRARY_PATH=/opt/soft/jdk/jre/lib/amd64/server
接下来继续运行2. [work@zc-mc-face01 build]# make -j4
3. [work@zc-mc-face01 build]# sudomakeinstall
4.临时修改环境变量[work@zc-mc-face01 build]# export LD_LIBRARY_PATH=/opt/glibc-2.14/lib:$LD_LIBRARY_PATH
5.接下来就可以正常启动redis了[work@zc-mc-face01 build]# cd ~/app/redis-2.8.13/src
[work@zc-mc-face01 src]# ./redis-server
./bin/redis-trib.rb create --replicas 1 192.168.5.155:6380 192.168.5.155:6381 192.168.5.155:6382 192.168.5.144:6380 192.168.5.144:6381 192.168.5.144:6382[root@node00 src]# ./redis-trib.rb add-node --slave --master-id4f6424e47a2275d2b7696bfbf8588e8c4c3a5b95 172.168.63.202:7001172.168.63.202:7000......[OK] All nodes agree about slotsconfiguration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.Connecting to node 172.168.63.202:7001: OK[ERR] Node 172.168.63.202:7001 is not empty. Either the nodealreadyknows other nodes (check with CLUSTER NODES) or contains somekey in database 0.解决方法：1)、将需要新增的节点下aof、rdb等本地备份文件删除；2)、同时将新Node的集群配置文件删除,即：删除你redis.conf里面cluster-config-file所在的文件；3)、再次添加新节点如果还是报错，则登录新Node,./redis-cli–hx –p对数据库进行清除：172.168.63.201:7001> flushdb #清空当前数据库 ./bin/redis-server ./conf/redis-6380.conf ; ./bin/redis-server ./conf/redis-6381.conf ; ./bin/redis-server ./conf/redis-6382.conf ;
