您的位置：首页 > 数据库 > Redis

Logstash+Redis+Elasticsearch+Kibana 快速搭建Nginx日志查询系统

2014-03-28 18:29 956 查看


Logstash+Redis+Elasticsearch+Kibana Nginx日志查询系统

环境
[code]elasticsearch-0.90.5.zip
kibana-latest.zip
redis-2.6.16.tar.gz
logstash-1.2.2-flatjar.jar


nginx.conf配置
[code]log_format  main  '$remote_addr - $remote_user [$time_local] ' 
                           '"$request" $status $body_bytes_sent '
                           '"$http_referer" "$http_user_agent" ';


nginx日志
[code]172.16.201.174 - - [25/Mar/2014:16:39:13 +0800] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1772.0 Safari/537.36"


表达式
[code]%{IPORHOST:source_ip} - %{USERNAME:remote_user} \[%{HTTPDATE:timestamp}\] %{QS:request} %{INT:status} %{INT:body_bytes_sent} %{QS:http_referer} %{QS:http_user_agent}


collection
[code]hadoop@stormspark:~/log/logstash$ cat  sp.conf 
input {
   file {
       type => "nginx-access"
       path => "/var/log/nginx/access.log"
   }
}
output {
    stdout {
        debug => true
        debug_format => json
    }

    redis {
        host => "127.0.0.1"
        port => 6379
        data_type => "list"
        key => "logstash"
    }
}


index配置
[code]hadoop@stormspark:~/log/logstash$ cat index.conf 
input {
  redis {
    host => "127.0.0.1"
    port => "6379"
    data_type => "list"
    key => "logstash"
    type => "redis-input"
  }
}
filter {
   grok {
       type => "nginx-access"
       pattern => "%{IPORHOST:source_ip} - %{USERNAME:remote_user} \[%{HTTPDATE:timestamp}\] %{QS:request} %{INT:status} %{INT:body_bytes_sent} %{QS:http_referer} %{QS:http_user_agent}"
   }
}
output {
  elasticsearch {
    host => "127.0.0.1"
  }
}


分别启动logstash,redis,es等。
[code]java -jar logstash-1.2.2-flatjar.jar agent -f sp.conf
java -jar logstash-1.2.2-flatjar.jar agent -f index.conf


最后来个截图：
内容来自用户分享和网络整理，不保证内容的准确性，如有侵权内容，可联系管理员处理 点击这里给我发消息
标签： 
相关文章推荐
新的分享
章节导航