nginx-access.log的logstash解析
1、nginx的access的格式:
log_format hehe '$http_host $server_addr $remote_addr [$time_local] "$request" $request_body $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_time $upstream_response_time $upstream_addr';
2、nginx日志
evo.xinpinget.com 10.1.217.143 6.52.65.86 [22/May/2018:15:10:46 +0800] "GET /api/merchant/listForSelection?filterByUserSelector=true&token=1cmqfl2RiO9BV9xIU8F3LG1I HTTP/1.1" - 200 268 "http://evo.xinpinget.com/order/newAfterSales" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.75 Safari/537.36 QQBrowser/4.1.4132.400" 0.021 0.021 10.1.132.201:4010
3、grok
%{IPORHOST:http_host} %{IPORHOST:server_addr} %{IPORHOST:remote_addr} \[%{HTTPDATE:time_local}\] "%{WORD:method} %{URIPATH:uri}\?%{NOTSPACE:params} HTTP/%{NUMBER:http_version}" %{NOTSPACE:request_body|-} %{INT:status} %{INT:body_bytes_sent} %{QS:http_referer} %{QS:http_user_agent} %{NUMBER:request_time:float} %{NUMBER:upstream_response_time:float} %{IPORHOST:upstream_addr}
4、logstash
patterns:
# cat nginx
HOSTPORT1 (%{IPV4}:%{POSINT}[, ]{0,2})+
NGINXACCESS %{IPORHOST:http_host} %{IPORHOST:server_addr} %{IPORHOST:remote_addr} \[%{HTTPDATE:time_local}\] "%{WORD:method} %{URIPATH:uri}\?%{NOTSPACE:params} HTTP/%{NUMBER:http_version}" %{NOTSPACE:request_body|-} %{INT:status} %{INT:body_bytes_sent} %{QS:http_referer} %{QS:http_user_agent} %{NUMBER:request_time:float} %{NUMBER:upstream_response_time:float} %{IPORHOST:upstream_addr}
logstash.conf
filter{
if 'access' in [tags] {
grok {
patterns_dir => "/etc/logstash/patters.d"
match =>{ "message" => "%{NGINXACCESS}"
overwrite => [ "message"]
}
}
kv {
source => "params"
field_split => "&?"
value_split => "="
include_keys => [ "token", "user", "from", "channelId" ]
}
}
}
- Python重写Logstash,把Nginx Access Log清洗后汇入Elastic DB
- logstash处理nginx accesslog
- nginx args urldecode 及nginx access_log中双引号被解析为x22问题解决方案
- Nginx访问日志(access_log)配置及信息详解
- apache/nginx access.log 说明
- nginx服务器access_log日志分析及配置详解
- shell分析nginx access log日志
- nginx access log logrotate配置
- 定时备份mysql, 定时切割nginx access log的方法
- logstash解析haproxy的haproxy.log日志字段
- Nginx Access Log日志统计分析常用命令
- nginx服务器access_log日志分析及配置详解
- nginx服务器access_log日志分析及配置详解
- nginx 出现不该出现的 access.log 日志
- nginx服务器access_log日志分析及配置详解
- nginx 完全关闭 access_log
- nginx的access.log文件详解,main的具体意思
- apache 访问日志access_log 配置和解析 rotatelogs分割日志
- Nginx修改access.log日志时间格式