ELK实战之收集Nginx的json格式日志
2017-12-27 16:50
846 查看
一、安装Nginx
[root@linux-node1 ~]# yum install nginx -y [root@linux-node1 ~]# vim /etc/nginx/nginx.conf #修改日志格式为json格式,并创建一个nginxweb的网站目录 log_format access_json '{"@timestamp":"$time_iso8601",' '"host":"$server_addr",' '"clientip":"$remote_addr",' '"size":$body_bytes_sent,' '"responsetime":$request_time,' '"upstreamtime":"$upstream_response_time",' '"upstreamhost":"$upstream_addr",' '"http_host":"$host",' '"url":"$uri",' '"domain":"$host",' '"xff":"$http_x_forwarded_for",' '"referer":"$http_referer",' '"status":"$status"}'; access_log /var/log/nginx/access.log access_json; location /nginxweb { root html; index index.html index.htm; } [root@linux-node1 ~]# mkdir /usr/share/nginx/html/nginxweb [root@linux-node1 ~]# echo "<h1> welcome to use Nginx" > /usr/share/nginx/html/nginxweb/index.html [root@linux-node1 ~]# nginx -t nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful [root@linux-node1 ~]# systemctl start nginx
二、配置logstash
[root@linux-node1 ~]# vim /etc/logstash/conf.d/nginx-accesslog.conf input{ file { path => "/var/log/nginx/access.log" type => "nginx-access-log" start_position => "beginning" stat_interval => "2" } } output{ elasticsearch { hosts => ["192.168.56.11:9200"] index => "logstash-nginx-access-log-%{+YYYY.MM.dd}" } file { path => "/tmp/logstash-nginx-access-log-%{+YYYY.MM.dd}" } } [root@linux-node1 ~]# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/nginx_access.conf -t [root@linux-node1 ~]# systemctl restart logstash
三、配置Kibana展示
[root@linux-node1 ~]# ab -n1000 -c 100 http://192.168.56.11/nginxweb/index.html #对页面压测 [root@linux-node1 ~]# tailf /var/log/nginx/access.log #nginx的访问日志变成了json格式 {"@timestamp":"2017-12-27T16:38:17+08:00","host":"192.168.56.11","clientip":"192.168.56.11","size":26,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"192.168.56.11","url":"/nginxweb/index.html","domain":"192.168.56.11","xff":"-","referer":"-","status":"200"} {"@timestamp":"2017-12-27T16:38:17+08:00","host":"192.168.56.11","clientip":"192.168.56.11","size":26,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"192.168.56.11","url":"/nginxweb/index.html","domain":"192.168.56.11","xff":"-","referer":"-","status":"200"} {"@timestamp":"2017-12-27T16:38:17+08:00","host":"192.168.56.11","clientip":"192.168.56.11","size":26,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"192.168.56.11","url":"/nginxweb/index.html","domain":"192.168.56.11","xff":"-","referer":"-","status":"200"} {"@timestamp":"2017-12-27T16:38:17+08:00","host":"192.168.56.11","clientip":"192.168.56.11","size":26,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"192.168.56.11","url":"/nginxweb/index.html","domain":"192.168.56.11","xff":"-","referer":"-","status":"200"} {"@timestamp":"2017-12-27T16:38:17+08:00","host":"192.168.56.11","clientip":"192.168.56.11","size":26,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"192.168.56.11","url":"/nginxweb/index.html","domain":"192.168.56.11","xff":"-","referer":"-","status":"200"} {"@timestamp":"2017-12-27T16:38:17+08:00","host":"192.168.56.11","clientip":"192.168.56.11","size":26,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"192.168.56.11","url":"/nginxweb/index.html","domain":"192.168.56.11","xff":"-","referer":"-","status":"200"} {"@timestamp":"2017-12-27T16:38:17+08:00","host":"192.168.56.11","clientip":"192.168.56.11","size":26,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"192.168.56.11","url":"/nginxweb/index.html","domain":"192.168.56.11","xff":"-","referer":"-","status":"200"} {"@timestamp":"2017-12-27T16:38:17+08:00","host":"192.168.56.11","clientip":"192.168.56.11","size":26,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"192.168.56.11","url":"/nginxweb/index.html","domain":"192.168.56.11","xff":"-","referer":"-","status":"200"} {"@timestamp":"2017-12-27T16:38:17+08:00","host":"192.168.56.11","clientip":"192.168.56.11","size":26,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"192.168.56.11","url":"/nginxweb/index.html","domain":"192.168.56.11","xff":"-","referer":"-","status":"200"} {"@timestamp":"2017-12-27T16:38:17+08:00","host":"192.168.56.11","clientip":"192.168.56.11","size":26,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"192.168.56.11","url":"/nginxweb/index.html","domain":"192.168.56.11","xff":"-","referer":"-","status":"200"}
Head插件查看:
Kibana查看:
相关文章推荐
- elk-nginx输出json格式的日志
- ELK实战之Tomcat的json日志收集
- elk系列3之通过json格式采集Nginx日志
- elk系列3之通过json格式采集Nginx日志【转】
- elk系统搭建并收集nginx日志-主要步骤
- Logstash收集json格式日志文件如何写配置文件
- ELK+syslog+nginx访问日志收集+分词处理
- Nginx 日志改成 JSON 格式
- ELK实战二:日志收集利器rsyslog
- elk实战分析nginx日志文档
- ELK实战之通过TCP收集日志
- ELK系列~Nxlog日志收集加转发(解决log4日志换行导致json转换失败问题)
- ELK实战之使用filebeat代替logstash收集日志
- Nginx 日志改成 JSON 格式的方法
- ELK收集Nginx日志,使用grok正则表达式(二)
- ELK Stack (2) —— ELK + Redis收集Nginx日志
- nginx中用JSON格式记录日志的配置示例
- ELK集群部署及收集nginx日志
- ELK实战-Logstash:收集rsyslog日志