logstash5.5.2部署-03

ubuntu16.04

1、安装前必须有Java

java -version
java version "1.8.0_65"
Java(TM) SE Runtime Environment (build 1.8.0_65-b17) Java HotSpot(TM) 64-Bit Server VM (build 25.65-b01, mixed mode)

2、apt

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
sudo apt-get install apt-transport-https
echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list

3、安装logstash

sudo apt-get update && sudo apt-get install logstash

centos系列

1、YUM

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch 
vi /etc/yum.repos.d/logstash.repo
[logstash-5.x]
name=Elastic repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

2、安装

sudo yum install logstash

测试

hello world

在客户端中测试，此处的客户端是ubuntu16.04系统

1、基本的输入输出

cd /usr/share/logstash/
bin/logstash -e 'input{stdin{}}output{stdout{codec=>rubydebug}}'

执行完命令之后会有以下报错，可忽略

ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.

WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using –path.settings. Continuing using the defaults

输入hello world

输出如下：

hello world
{
"@timestamp" => 2017-08-08T06:13:34.505Z,
"@version" => "1",
"host" => "node1",
"message" => "hello world"
}

2、把内容写到elasticsearch中

root@ncnode03:/usr/share/logstash# pwd
/usr/share/logstash
root@ncnode03:/usr/share/logstash# bin/logstash -e 'input { stdin{} } output { elasticsearch { hosts => ["192.168.96.208:9200"]} }'
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path //usr/share/logstash/config/log4j2.properties. Using default config which logs to console
10:22:50.006 [[main]-pipeline-manager] INFO  logstash.outputs.elasticsearch - Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://192.168.96.208:9200/]}}
10:22:50.010 [[main]-pipeline-manager] INFO  logstash.outputs.elasticsearch - Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://192.168.96.208:9200/, :path=>"/"}
10:22:50.130 [[main]-pipeline-manager] WARN  logstash.outputs.elasticsearch - Restored connection to ES instance {:url=>"http://192.168.96.208:9200/"}
10:22:50.132 [[main]-pipeline-manager] INFO  logstash.outputs.elasticsearch - Using mapping template from {:path=>nil}
10:22:50.303 [[main]-pipeline-manager] INFO  logstash.outputs.elasticsearch - Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
10:22:50.312 [[main]-pipeline-manager] INFO  logstash.outputs.elasticsearch - New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//192.168.96.208:9200"]}
10:22:50.318 [[main]-pipeline-manager] INFO  logstash.pipeline - Starting pipeline {"id"=>"main", "pipeline.workers"=>32, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>4000}
10:22:50.356 [[main]-pipeline-manager] INFO  logstash.pipeline - Pipeline main started
The stdin plugin is now waiting for input:
10:22:50.396 [Api Webserver] INFO  logstash.agent - Successfully started Logstash API endpoint {:port=>9600}
you shi yi ge zhou mo   #此处为输入的信息

去es网页上查看，数据是否同步



去kibana上查看，数据是否同步



此处只是一个简单的测试，关于logstash自动收集nginx、tomcat或者像openstack的Nova日志、glance日志、neutron日志等后续会继续完善，欢迎大家互相学习，如发现文章哪里有问题，请及时指正，谢谢

logstash官网图片



logstash收集日志主要是编辑pipeline.conf文件

更多logstash内容可参考https://kibana.logstash.es/content/logstash/get-start/full-config.html

———————————年轻的时候最幸福的事情就是拼命工作——————————–