docker-compose快速搭建 Prometheus+Grafana监控系统
一、说明
Prometheus负责收集数据,Grafana负责展示数据。其中采用Prometheus 中的 Exporter含:
1)Node Exporter,负责收集 host 硬件和操作系统数据。它将以容器方式运行在所有 host 上。
2)cAdvisor,负责收集容器数据。它将以容器方式运行在所有 host 上。
3)Alertmanager,负责告警。它将以容器方式运行在所有 host 上。
完整Exporter列表请参考:https://prometheus.io/docs/instrumenting/exporters/
二、安装docker,docker-compose
2.1 安装docker
先安装一个64位的Linux主机,其内核必须高于3.10,内存不低于1GB。在该主机上安装Docker。
安装依赖包
yum install -y yum-utils device-mapper-persistent-data lvm2
添加Docker软件包源
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
安装Docker CE
yum install docker-ce -y
启动
systemctl start docker
开机启动
systemctl enable docker
查看Docker信息
docker info
2.2 安装docker-compose
curl -L https://github.com/docker/compose/releases/download/1.23.2/docker-compose-
uname -s-
uname -m-o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
三、添加配置文件
mkdir -p /usr/local/src/config
cd /usr/local/src/config
2.1 添加prometheus.yml配置文件,
vim prometheus.yml
my global config
global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
scrape_timeout is set to the global default (10s).
Alertmanager configuration
alerting:
alertmanagers:
- static_configs:
targets: [‘192.168.159.129:9093’]
- alertmanager:9093
Load rules once and periodically evaluate them according to the global ‘evaluation_interval’.
rule_files:
- “node_down.yml”
- “first_rules.yml”
- “second_rules.yml”
A scrape configuration containing exactly one endpoint to scrape:
Here it’s Prometheus itself.
scrape_configs:
The job name is added as a label job=<job_name>
to any timeseries scraped from this config.
-
job_name: ‘prometheus’
static_configs: -
targets: [‘192.168.159.129:9090’]
job_name: ‘cadvisor’
targets: [‘192.168.159.129:8080’]
static_configs:
job_name: ‘node’
scrape_interval: 8s
static_configs:
-
targets: [‘192.168.159.129:9100’]
2.2 添加邮件告警配置文件
添加配置文件alertmanager.yml,配置收发邮件邮箱
vim alertmanager.yml
global:
smtp_smarthost: ‘smtp.163.com:25’ #163服务器
smtp_from: ‘tsiyuetian@163.com’ #发邮件的邮箱
smtp_auth_username: ‘tsiyuetian@163.com’ #发邮件的邮箱用户名,也就是你的邮箱
smtp_auth_password: ‘TPP***’ #发邮件的邮箱密码
smtp_require_tls: false #不进行tls验证
route:
group_by: [‘alertname’]
group_wait: 10s
group_interval: 10s
repeat_interval: 10m
receiver: live-monitoring
receivers:
- name: ‘live-monitoring’
email_configs: to: ‘1933306137@qq.com’ #收邮件的邮箱
2.3 添加报警规则
添加一个node_down.yml为 prometheus targets 监控
vim node_down.yml
groups:
- name: node_down
rules: alert: InstanceDown
expr: up == 0
for: 1m
labels:
user: test
annotations:
summary: “Instance {{ $labels.instance }} down”
description: “{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 1 minutes.”
四、编写docker-compose
vim docker-compose-monitor.yml version: '2' networks: monitor: driver: bridge services: prometheus: image: prom/prometheus container_name: prometheus hostname: prometheus restart: always volumes: - /usr/local/src/config/prometheus.yml:/etc/prometheus/prometheus.yml - /usr/local/src/config/node_down.yml:/etc/prometheus/node_down.yml ports: - "9090:9090" networks: - monitor alertmanager: image: prom/alertmanager container_name: alertmanager hostname: alertmanager restart: always volumes: - /usr/local/src/config/alertmanager.yml:/etc/alertmanager/alertmanager.yml ports: - "9093:9093" networks: - monitor grafana: image: grafana/grafana container_name: grafana hostname: grafana restart: always ports: - "3000:3000" networks: - monitor node-exporter: image: quay.io/prometheus/node-exporter container_name: node-exporter hostname: node-exporter restart: always ports: - "9100:9100" networks: - monitor cadvisor: image: google/cadvisor:latest container_name: cadvisor hostname: cadvisor restart: always volumes: - /:/rootfs:ro - /var/run:/var/run:rw - /sys:/sys:ro - /var/lib/docker/:/var/lib/docker:ro ports: - "8080:8080" networks: - monitor
五、启动docker-compose
#启动容器: docker-compose -f /usr/local/src/config/docker-compose-monitor.yml up -d #删除容器: docker-compose -f /usr/local/src/config/docker-compose-monitor.yml down #重启容器: docker restart id 容器启动如下: docker-compose快速搭建 Prometheus+Grafana监控系统
prometheus targets界面如下:
docker-compose快速搭建 Prometheus+Grafana监控系统
备注:如果State为Down,应该是防火墙问题,参考下面防火墙配置。
prometheus graph界面如下:
docker-compose快速搭建 Prometheus+Grafana监控系统
备注:如果没有数据,同步下时间。
六、防火墙配置
6.1 关闭selinux
setenforce 0 vim /etc/sysconfig/selinux 6.2 配置iptables #删除自带防火墙 systemctl stop firewalld.service systemctl disable firewalld.service #安装iptables yum install -y iptables-services #配置 vim /etc/sysconfig/iptables *filter :INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [24:11326] -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 9090 -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 8080 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 3000 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 9093 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 9100 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT
#启动 systemctl restart iptables.service systemctl enable iptables.service
七、配置Grafana
7.1 添加Prometheus数据源
2 配置dashboards
说明:可以用自带模板,也可以去https://grafana.com/dashboards,下载对应的模板。
7.3 查看数据
我从网页下载了docker相关的模板:Docker and system monitoring,893
输入893,就会加载出下面的信息
导入后去首页查看数据
八、附录:单独命令启动各容器
#启动prometheus docker run -d -p 9090:9090 --name=prometheus \ -v /usr/local/src/config/prometheus.yml:/etc/prometheus/prometheus.yml \ -v /usr/local/src/config/node_down.yml:/etc/prometheus/node_down.yml \ prom/prometheus
启动grafana
docker run -d -p 3000:3000 --name=grafana grafana/grafana
#启动alertmanager容器 docker run -d -p 9093:9093 -v /usr/local/src/config/config.yml:/etc/alertmanager/config.yml --name alertmanager prom/alertmanager #启动node exporter docker run -d \ -p 9100:9100 \ -v "/:/host:ro,rslave" \ --name=node_exporter \ quay.io/prometheus/node-exporter \ --path.rootfs /host #启动cadvisor docker run \ --volume=/:/rootfs:ro \ --volume=/var/run:/var/run:rw \ --volume=/sys:/sys:ro \ --volume=/var/lib/docker/:/var/lib/docker:ro \ --publish=8080:8080 \ --detach=true \ --name=cadvisor \ google/cadvisor:latest
欢迎工作一到五年的Java工程师朋友们加入Java高并发: 957734884,群内提供免费的Java架构学习资料(里面有高可用、高并发、高性能及分布式、Jvm性能调优、Spring源码,MyBatis,Netty,Redis,Kafka,Mysql,Zookeeper,Tomcat,Docker,Dubbo,Nginx等多个知识点的架构资料)合理利用自己每一分每一秒的时间来学习提升自己,不要再用"没有时间“来掩饰自己思想上的懒惰!趁年轻,使劲拼,给未来的自己一个交代!
- docker-compose快速搭建 Prometheus+Grafana监控系统
- docker:快速构建容器监控系统cAdvisor+InfluxDB+Grafana
- 使用influxdb+cadvisor+grafana的docker镜像搭建一个实时监控系统的环境
- docker:快速构建容器监控系统cAdvisor+InfluxDB+Grafana
- 使用 Docker 部署 Grafana + Prometheus 监控 MySQL 数据库
- 从零开始搭建Prometheus自动监控报警系统
- centos 快速搭建lnmp、lamp+zabbix3.2监控系统
- Windows下本机简易监控系统搭建(Telegraf+Influxdb+Grafana)--转
- Windows下本机简易监控系统搭建(Telegraf+Influxdb+Grafana)
- cAdvisor、InfluxDB、Grafana搭建Docker1.12性能监控平台
- Prometheus+Grafana搭建监控系统(一)
- 使用 StatsD + Grafana + InfluxDB 搭建 Node.js 监控系统
- Docker监控——Cadvisor+InfluxDB+Grafana搭建过程
- Prometheus+Grafana搭建监控系统
- Grafana3.0.1+Zabbix3.0.4监控系统平台搭建
- .NetCore下使用Prometheus实现系统监控和警报 (三)集成Grafana
- 通过docker搭建zabbix监控系统
- 使用StatsD, Graphite, Grafana, Kamon搭建可用于JVM项目的可视化性能监控系统
- 58.应用性能监控APM之collectd+InfluxDB+Grafana系统搭建
- 使用 docker-compose.yml 快速搭建php开发环境