您的位置：首页 > 运维架构 > Nginx

利用grok对Nginx日志进行分解

2017-03-09 15:29 369 查看

提前对Nginx日志的格式进行定义，根据定义好的日志进行分解

Nginx日志格式为：

192.168.1.106 192.168.1.106 101.246.116.62 "09/Mar/2017:15:18:59 +0800" "GET" "/xxxx/data/xxxx/hls/20160505/8cd7a8680d795007/FHD/seg-0031.ts" "-" 200 901836 "-" "AppleCoreMedia/1.0.0.9A405 (iPad; U; CPU OS 5_0_1 like Mac OS X; zh_cn)" "-" 0.334
192.168.1.106 192.168.1.106 101.246.116.62 "09/Mar/2017:15:19:00 +0800" "GET" "/xxxx/data/xxxx/hls/20160505/8cd7a8680d795007/FHD/seg-0032.ts" "-" 200 1257532 "-" "AppleCoreMedia/1.0.0.9A405 (iPad; U; CPU OS 5_0_1 like Mac OS X; zh_cn)" "-" 0.415

grok规则如下：

NGUSERNAME [a-zA-Z\.\@\-\+_%]+
NGUSER %{NGUSERNAME}
NGINXACCESS %{IP:host} %{IP:server_addr} %{IP:client} \"%{HTTPDATE:timestamp}\" \"%{WORD:method}\" \"%{URIPATHPARAM:request}\" "-" %{NUMBER:response} %{NUMBER:bytes} "-" %{QS:agent} "-" %{NUMBER:duration}

中间增加了Geo的ip定位功能，可以区分客户端是来自哪个地区，最后，日志分解如下：

{
"_index": "logstash-nginx-access-2017.03.09",
"_type": "nginx_access",
"_id": "AVqx9hOPoUu7TGFKA5Fz",
"_score": null,
"_source": {
"@version": "1",
"@timestamp": "2017-03-09T07:26:17.000Z",
"path": "/var/log/messages",
"host": [
"wh-01",
"192.168.1.106"
],
"server_addr": "192.168.1.106",
"client": "117.75.3.85",
"method": "GET",
"request": "/xxxx/data/xxxx/hls/20160514/5db03833349436df/FHD/seg-0095.ts",
"response": 200,
"bytes": 1432936,
"agent": "\"-\"",
"duration": "0.503",
"geoip": {
"ip": "117.75.3.85",
"country_code2": "CN",
"country_code3": "CHN",
"country_name": "China",
"continent_code": "AS",
"region_name": "22",
"city_name": "Beijing",
"latitude": 39.9289,
"longitude": 116.38830000000002,
"timezone": "Asia/Harbin",
"real_region_name": "Beijing",
"location": [
116.38830000000002,
39.9289
],
"coordinates": [
116.38830000000002,
39.9289
]
},
"type": "nginx_access"
},
"fields": {
"@timestamp": [
1489044377000
]
},
"sort": [
1489044377000
]
}

grok规则匹配可以去grok debugger进行测试匹配

内容来自用户分享和网络整理，不保证内容的准确性，如有侵权内容，可联系管理员处理

标签： grok nginx log

相关文章推荐

新的分享

章节导航