Centos7 安装ELK

1、概述

ELK 简介

ELK 是Elasticsearch+Logstash+Kibana的简称：

Elasticsearch是一个基于Lucene的搜索服务器。它提供了一个分布式多用户能力的全文搜索引擎，基于java开发

Logstash是一个接收，处理，转发日志的工具。

Kibana是一个基于浏览器页面的Elasticsearch前端展示工具。Kibana全部使用HTML语言和Javascript编写的

操作系统版本

cat /etc/redhat-release
CentOS Linux release 7.0.1406 (Core)

2、 系统配置

关闭selinux

sed -i "s/SELINUX=enforcing/SELINUX=disabled/" /etc/selinux/config
setenforce 0

设置firewall

安装firewall（若没有，先安装）

yum install firewalld firewall-config
systemctl start firewalld.service
systemctl enable firewalld.service
systemctl status firewalld

需要开放的端口

服务	需要开放的端口
Elasticsearch	tcp/9200和9300
kibana	tcp/5601
logstash	tcp/5000

firewall-cmd --permanent --add-port={9200/tcp,9300/tcp}
firewall-cmd --permanent --add-port=5601/tcp
firewall-cmd --permanent --add-port=5000/tcp
firewall-cmd --reload
firewall-cmd --state
firewall-cmd --list-all

设置FQND

#cat /etc/hostname
elk

#cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.0.102 elk.zll.com  elk

#hostname -F /etc/hostname

#hostname -f
elk.zll.com

3、安装Elasticsearch

yum install java-1.7.0-openjdk    (安装java)
wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.1.noarch.rpm yum  -y localinstall elasticsearch-1.7.1.noarch.rpm

启动服务

systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl start elasticsearch
systemctl status elasticsearch

查看Elasticsearch配置文件

rpm -qc elasticsearch
/etc/elasticsearch/elasticsearch.yml
/etc/elasticsearch/logging.yml
/etc/init.d/elasticsearch
/etc/sysconfig/elasticsearch
/usr/lib/sysctl.d/elasticsearch.conf
/usr/lib/systemd/system/elasticsearch.service
/usr/lib/tmpfiles.d/elasticsearch.conf

查看Elasticsearch日志文件



查看Elasticsearch端口（在firewall中放行）

4、安装kibana

下载软件包

wget https://download.elastic.co/kibana/kibana/kibana-4.1.1-linux-x64.tar.gz tar zxf kibana-4.1.1-linux-x64.tar.gz -C /usr/local/
cd /usr/local/
mv kibana-4.1.1-linux-x64 kibana

创建kibana.service启动文件。

cat > /etc/systemd/system/kibana.service <<EOF
[Service]
ExecStart=/usr/local/kibana/bin/kibana

[Install]
WantedBy=multi-user.target
EOF

启动kibana服务

systemctl enable kibana
systemctl start kibana
systemctl status kibana

查看kibana端口



web输入 http://ip_address:5601

5、安装Logstash

安装软件包

wget https://download.elastic.co/logstash/logstash/packages/centos/logstash-1.5.3-1.noarch.rpm yum localinstall logstash-1.5.3-1.noarch.rpm

设置ssl

使用FQDN创建SSL 证书（例：elk.zll.com）

cd /etc/pki/tls
openssl req -subj '/CN=elk.zll.com/' -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout private/logstash-forwarder.key -out certs/logstash-forwarder.crt

创建一个01-logstash-initial.conf 文件

cat > /etc/logstash/conf.d/01-logstash-initial.conf << EOF
input {
lumberjack {
port => 5000
type => "logs"
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}

filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}

output {
elasticsearch { host => localhost }
stdout { codec => rubydebug }
}
EOF

启动logstash服务

systemctl restart logstash
systemctl status logstash
chkconfig logstash on       (开机启动设置特殊)

查看logstash日志，是否有报错

tail /var/log/logstash/logstash.log

查看logstash端口(firewall中开启)

6、客户端安装Logstash Forwarder

安装软件包

wget https://download.elastic.co/logstash-forwarder/binaries/logstash-forwarder-0.4.0-1.x86_64.rpm yum localinstall logstash-forwarder-0.4.0-1.x86_64.rpm

修改配置文件/etc/logstash-forwarder.conf

修改配置文件中的elk-server

cp /etc/logstash-forwarder.conf /etc/logstash-forwarder.conf.old
cat > /etc/logstash-forwarder.conf << EOF
{
"network": {
"servers": [ "elk.zll.com:5000" ],

"ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt",

"timeout": 15
},

"files": [
{
"paths": [
"/var/log/messages",
"/var/log/secure"
],

"fields": { "type": "syslog" }
}
]
}
EOF

启动服务并设置开启启动

systemctl restart logstash-forwarder
chkconfig logstash-forwarder on
systemctl status logstash-forwarder

登录elk-server的web界面进行配置：http://ip_address:5601

本文参考：陈沙克日志