了解一下Logstash常用配置
2016-07-07 16:59
323 查看
Logstash管道可以配置一个或多个输入插件、过滤器插件和输出插件。其中,输入插件和输出插件是必选的,过滤器插件是可选的。下图是Logstash常见的使用场景。
上一节的例子中我们使用标准的输入和输出插件做了简单的示例。接下来我们演示一些复杂的场景。如下图所示是Logstash的标准管道结构,我们通过一些高级配置来完成Apache日志的过滤。
1. 准备一段apache日志文件,格式如下:
2. 编写Logstash管道配置文件,放在Logstash/bin目录下
3. 校验配置文件是否正确
4.启动Logstash
[root@Server05 bin]# ./logstash -f apache-log-pipeline.conf
Settings: Default pipeline workers: 4
Pipeline main started
5.完整的Logstash配置文件如下
上一节的例子中我们使用标准的输入和输出插件做了简单的示例。接下来我们演示一些复杂的场景。如下图所示是Logstash的标准管道结构,我们通过一些高级配置来完成Apache日志的过滤。
# The # character at the beginning of a line indicates a comment. Use# comments to describe your configuration. input { } # The filter part of this file is commented out to indicate that it is# optional. # filter { # # } output { } |
83.149.9.216 - - [04/Jan/2015:05:13:42 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-search.png HTTP/1.1" 200 203023 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36" 83.149.9.216 - - [04/Jan/2015:05:13:42 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-dashboard3.png HTTP/1.1" 200 171717 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36" 83.149.9.216 - - [04/Jan/2015:05:13:44 +0000] "GET /presentations/logstash-monitorama-2013/plugin/highlight/highlight.js HTTP/1.1" 200 26185 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36" 83.149.9.216 - - [04/Jan/2015:05:13:44 +0000] "GET /presentations/logstash-monitorama-2013/plugin/zoom-js/zoom.js HTTP/1.1" 200 7697 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36" |
input { file { path => "/opt/cx/logstash/apache-log.log" start_position => beginning } } filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}"} } geoip { source => "clientip" } } output { elasticsearch {} stdout {} }
3. 校验配置文件是否正确
[root@Server01 bin]# ./logstash -f apache-log-pipeline.conf --configtest Configuration OK
4.启动Logstash
[root@Server05 bin]# ./logstash -f apache-log-pipeline.conf
Settings: Default pipeline workers: 4
Pipeline main started
5.完整的Logstash配置文件如下
input { file { path => "/opt/cx/logstash/apache-log.log" start_position => beginning } } filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}"} } geoip { source => "clientip" } } output { elasticsearch { hosts=>["10.0.10.5:9200"] } stdout {} }
相关文章推荐
- 使用ElasticSearch+LogStash+Kibana+Redis搭建日志管理服务
- LogStash 安装配置初体验
- logStash收集日志并存储到Elasticsearch
- #研发解决方案#基于Apriori算法的Nginx+Lua+ELK异常流量拦截方案
- Logstash 与Elasticsearch整合使用示例
- ELK(ElasticSearch, Logstash, Kibana)搭建实时日志分析平台
- centos 7安装jdk和elk
- 企业级日志收集系统——ELKstack
- logstash
- logstash,elasticsearch,kibana三件套
- 用Kibana和logstash快速搭建实时日志查询、收集与分析系
- logstash
- 开源分布式搜索平台ELK+Redis+Syslog-ng实现日志实时搜索
- Loggly:提高ElasticSearch性能的九个高级配置技巧
- elk在centos7安装
- logstash过滤nginx日志
- 配置Logstash(1) — 配置文件的结构
- 配置Logstash(2) — “事件”相关配置
- Logstash扩展开发 - Input 与 Codec
- ELK 索引抽取模板(中文索引配置not_analyzed,才能在kibana中使用terms)