生产环境elasticsearch5.0.1集群的部署配置详解

线上环境elasticsearch5.0.1集群的配置部署

es集群的规划：

硬件：

7台8核、64G内存、2T ssd硬盘加1台8核16G的阿里云服务器

其中一台作为kibana+kafka连接查询的服务器

其他6台都作为node和master两种角色

操作系统：centos7.2 x86_64

为方便磁盘扩容建议将磁盘进行lvm逻辑卷配置，可以参考：

aliyun添加数据盘后的物理分区和lvm逻辑卷两种挂载方式
http://blog.csdn.net/reblue520/article/details/54174178

1.安装jdk1.8和elasticsearch5.0.1

rpm -ivh jdk-8u111-linux-x64.rpm

tar -zxvf elasticsearch-5.0.1.tar.gz

2.添加yunva这个运行elasticsearch的用户(es必须使用非root用户启动)

useradd yunva -d /home/yunva

echo 'pass'|passwd --stdin yunva

chown -R yunva.yunva /data

修改默认端口

sed -i 's/#Port 22/Port 2222/' /etc/ssh/sshd_config

sed -i 's/^GSSAPIAuthentication yes$/GSSAPIAuthentication no/' /etc/ssh/sshd_config

service sshd restart

3.针对es做的一些系统的优化配置

swapoff -a

echo "fs.file-max = 1000000" >> /etc/sysctl.conf

echo "vm.max_map_count=262144" >> /etc/sysctl.conf

echo "vm.swappiness = 1" >> /etc/sysctl.conf

sysctl -p

sed -i 's/* soft nofile 65535/* soft nofile 655350/g' /etc/security/limits.conf

sed -i 's/* hard nofile 65535/* hard nofile 655350/g' /etc/security/limits.conf

将java_home加入环境变量

cat >> /etc/profile <<EOF

export JAVA_HOME=/usr/java/jdk1.8.0_111

export PATH=\$JAVA_HOME/bin:\$PATH

EOF

source /etc/profile

4.es内存调整配置文件(建议配置为物理内存的一半或者更多最好不要超过32G，超过了也可能不会增强性能)：

/data/elasticsearch-5.0.1/config/jvm.options

sed -i 's/-Xms2g/-Xms32g/' /data/elasticsearch-5.0.1/config/jvm.options

sed -i 's/-Xmx2g/-Xmx32g/' /data/elasticsearch-5.0.1/config/jvm.options

echo "-Xss256k" >>/data/elasticsearch-5.0.1/config/jvm.options

sed -i 's/-XX:+UseConcMarkSweepGC/-XX:+UseG1GC/' /data/elasticsearch-5.0.1/config/jvm.options

5.集群的主要配置文件

修改elasticsearch的参数

vim /etc/elasticsearch/elasticsearch.yml(rpm安装方式的配置文件位置)

vim /data/elasticsearch-5.0.1/config/elasticsearch.yml

es节点的配置：

# 节点名

cluster.name: yunva-es

# 集群的名称，可以不写

discovery.zen.ping.unicast.hosts: ["node-1","yunva_etl_es2", "yunva_etl_es3","yunva_etl_es4","yunva_etl_es5","yunva_etl_es6","yunva_etl_es7"]

node.name: yunva_etl_es6

node.master: true

node.data: true

path.data: /data/es/data

path.logs: /data/es/logs

action.auto_create_index: false

indices.fielddata.cache.size: 12g

bootstrap.memory_lock: false

# 内网地址，可以加快速度

network.host: 192.168.1.10

http.port: 9200

# 增加新的参数head插件可以访问es

http.cors.enabled: true

http.cors.allow-origin: "*"

gateway.recover_after_time: 8m

gateway.expected_nodes: 3

cluster.routing.allocation.node_initial_primaries_recoveries: 8

# 以下配置可以减少当es节点短时间宕机或重启时shards重新分布带来的磁盘io读写浪费
discovery.zen.fd.ping_timeout: 180s
discovery.zen.fd.ping_retries: 8
discovery.zen.fd.ping_interval: 30s
discovery.zen.ping_timeout: 120s

针对kibana的es配置(非node和master节点)

# cat /etc/elasticsearch/elasticsearch.yml

cluster.name: yunva-es

node.name: yunva_etl_es1

node.master: false

node.data: false

node.ingest: false

action.auto_create_index: false

path.data: /data/es/data

path.logs: /data/es/logs

bootstrap.memory_lock: false

network.host: 0.0.0.0

http.port: 9200

http.cors.enabled: true

http.cors.allow-origin: "*"

# 以下配置可以减少当es节点短时间宕机或重启时shards重新分布带来的磁盘io读写浪费
discovery.zen.fd.ping_timeout: 180s
discovery.zen.fd.ping_retries: 8
discovery.zen.fd.ping_interval: 30s
discovery.zen.ping_timeout: 120s

注意修改配置文件vim /etc/hosts 列出集群节点名称和对应ip地址的对应关系(有内网dns并且配置的就不需要再次配置了)

echo "10.28.50.131 node-1" >> /etc/hosts

echo "10.26.241.239 yunva_etl_es3" >> /etc/hosts

echo "10.25.135.215 yunva_etl_es2" >> /etc/hosts

echo "10.26.241.237 yunva_etl_es4" >> /etc/hosts

echo "10.27.78.228 yunva_etl_es5" >> /etc/hosts

echo "10.27.65.121 yunva_etl_es6" >> /etc/hosts

echo "10.27.35.94 yunva_etl_es7" >> /etc/hosts

6.创建日志和数据存放目录

mkdir -p /data/es/data

mkdir /data/es/logs

chown -R yunva.yunva /data

7.启动es服务：

# su - yunva

[yunva]$ cd /data/elasticsearch-5.0.1/bin/

./elasticsearch &

8.检查单台服务是否正常：

$ curl http://ip:9200/
{

"name" : "yunva_etl_es5",

"cluster_name" : "yunva-es",

"cluster_uuid" : "2shAg8u3SjCRNJ4mEUBzBQ",

"version" : {

"number" : "5.0.1",

"build_hash" : "080bb47",

"build_date" : "2016-11-11T22:08:49.812Z",

"build_snapshot" : false,

"lucene_version" : "6.2.1"

},

"tagline" : "You Know, for Search"

}

# 查看集群状态

$ curl http://ip:9200/_cluster/health/?pretty
{

"cluster_name" : "yunva-es",

"status" : "green",

"timed_out" : false,

"number_of_nodes" : 5,

"number_of_data_nodes" : 4,

"active_primary_shards" : 66,

"active_shards" : 132,

"relocating_shards" : 2,

"initializing_shards" : 0,

"unassigned_shards" : 0,

"delayed_unassigned_shards" : 0,

"number_of_pending_tasks" : 0,

"number_of_in_flight_fetch" : 0,

"task_max_waiting_in_queue_millis" : 0,

"active_shards_percent_as_number" : 100.0

}

然后将配置好的es程序拷贝到其他服务器中，注意修改以下内容(network.host为内网地址，速度更快，节省互相复制、分片的时候处理带宽)：

1.elasticsearch.yml文件的配置修改

node.name: 节点名称

network.host: es节点的内网IP地址

2./etc/hosts文件中内网ip和node.name的对应关系

后续添加对集群服务的监控，可以参考：