mac 环境安装日志收集系统搭建(Fluent,ES,Kibana)

1.安装fluent：
https://www.fluentd.org/download
选择 Mac环境安装

安装成功后，测试一下

$ sudo launchctl load /Library/LaunchDaemons/td-agent.plist
$ less /var/log/td-agent/td-agent.log

$ curl -X POST -d 'json={"json":"message"}' http://localhost:8888/debug.test $ tail -n 1 /var/log/td-agent/td-agent.log

2013-04-19 16:51:47 -0700 debug.test: {"json":"message"}

配置文件

suo vim /etc/td-agent/td-agent.conf 
＃数据源通过tcp协议

<source>
  @type forward
  port 24224
</source>
＃匹配到tag,将数据存储到elasticsearch里

<match myapp.*>
  @type elasticsearch
  host 127.0.0.1
  port 9200
  index fluent_log
  type fluent
  logstash_format true
  flush_interval 10s
</match>
下载Elasticsearch 地址:https://www.elastic.co/downloads/elasticsearch
cd elasticsearch-5.4.1

./bin/elasticsearch
测试一下
curl 127.0.0.1:9200
返回结果：

{
  "name" : "Ryb91qD",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "G3UFMzGiTzmR1QUp4viR_A",
  "version" : {
    "number" : "5.4.1",
    "build_hash" : "2cfe0df",
    "build_date" : "2017-05-29T16:05:51.443Z",
    "build_snapshot" : false,
    "lucene_version" : "6.5.1"
  },
  "tagline" : "You Know, for Search"
}

修改es配置文件
添加

http.cors.enabled: true  #是否支持跨域，默认为false
http.cors.allow-origin: "*"   #当设置允许跨域，默认为*,表示支持所有域名

插件下载地址：https://github.com/mobz/elasticsearch-head
git clone git://github.com/mobz/elasticsearch-head.git
cd elasticsearch-head
npm install（电脑提前得安装node）
npm run start

open

 http://localhost:9100/

安装fluent 操作es的插件:

/opt/td-agent/

embedded/bin/gem install
fluent-plugin-elasticsearch-1.0.0.gem
依赖包

elasticsearch-1.0.12.gem
elasticsearch-api-1.0.12.gem
elasticsearch-transport-1.0.12.gem
excon-0.45.4.gem
faraday-0.9.1.gem
fluent-plugin-elasticsearch-1.0.0.gem
multi_json-1.11.2.gem
multipart-post-2.0.0.gem

安装kibana

https://www.elastic.co/downloads/kibana
编辑kibana/config/kibana.yml，把elasticsearch.url注释打开

bin/kibana

curl 127.0.0.1:5601