《kubernetes-1.8.0》09-addon-kube-prometheus

《kubernetes-1.8.0》09-addon-kube-prometheus

一、环境准备

git clone https://github.com/coreos/prometheus-operator.git[/code] 
指定kubeconfig目录位置（本例使用缺省，如果配置文件存放位置不是~/.kube/config需指定该变量）：

export KUBECONFIG=<path> # defaults to "~/.kube/config"


二、安装kube-prometheus
运行kube-prometheus部署脚本：

$ cd prometheus-operator/contrib/kube-prometheus/
$ hack/cluster-monitoring/deploy
...
namespace "monitoring" created
clusterrolebinding "prometheus-operator" created
clusterrole "prometheus-operator" created
serviceaccount "prometheus-operator" created
service "prometheus-operator" created
deployment "prometheus-operator" created
Waiting for Operator to register custom resource definitions....................
daemonset "node-exporter" created
service "node-exporter" created
clusterrolebinding "kube-state-metrics" created
clusterrole "kube-state-metrics" created
deployment "kube-state-metrics" created
rolebinding "kube-state-metrics" created
role "kube-state-metrics-resizer" created
serviceaccount "kube-state-metrics" created
service "kube-state-metrics" created
secret "grafana-credentials" created
secret "grafana-credentials" configured
configmap "grafana-dashboards-0" created
deployment "grafana" created
service "grafana" created
servicemonitor "kube-state-metrics" created
servicemonitor "prometheus-operator" created
prometheus "k8s" created
servicemonitor "kubelet" created
servicemonitor "kube-controller-manager" created
serviceaccount "prometheus-k8s" created
servicemonitor "prometheus" created
servicemonitor "kube-apiserver" created
servicemonitor "alertmanager" created
servicemonitor "kube-scheduler" created
configmap "prometheus-k8s-rules" created
servicemonitor "node-exporter" created
service "prometheus-k8s" created
role "prometheus-k8s" created
role "prometheus-k8s" created
role "prometheus-k8s" created
clusterrole "prometheus-k8s" created
rolebinding "prometheus-k8s" created
rolebinding "prometheus-k8s" created
rolebinding "prometheus-k8s" created
clusterrolebinding "prometheus-k8s" created
secret "alertmanager-main" created
service "alertmanager-main" created
alertmanager "main" created


创建pod时间较长，镜像需要下比较久，其中 
addon-resizer:1.0
 在之前的镜像包中提供。

一个小插曲：

加载完毕后，30900端口访问失败，怀疑prometheus核心组件启动失败，但检查pod又都是Running。

查看日志后发现：

kube-controller-manage
报错：

Event(v1.ObjectReference{Kind:”StatefulSet”, Namespace:”monitoring”, Name:”prometheus-k8s”, UID:”2fbbb615-d117-11e7-917e-005056bc52e5”, APIVersion:”apps”, ResourceVersion:”256502”, FieldPath:”“}): type: ‘Warning’ reason: ‘FailedCreate’ create Pod prometheus-k8s-0 in StatefulSet prometheus-k8s failed error: pods “prometheus-k8s-0” is forbidden: pod.Spec.SecurityContext.RunAsUser is forbidden

检查statefulset启动情况：



发现
prometheus-k8s
这个statefulset没有成功启动：

用describe查看：



排查半天，发觉可能是apiserver 的控制问题，检查apiserver配置文件：

去除：
KUBE_ADMISSION_CONTROL
中的
SecurityContextDeny
部分，分别重启apiserver。

再次检查statefulset和pod的情况：

[root@node-131 kube-prometheus]# kubectl get pods -n monitoring
NAME                                   READY     STATUS    RESTARTS   AGE
alertmanager-main-0                    2/2       Running   0          16m
alertmanager-main-1                    2/2       Running   0          15m
alertmanager-main-2                    2/2       Running   0          15m
grafana-7d966ff57-twzx9                2/2       Running   0          17m
kube-state-metrics-b5f8d6c5-zvgh5      2/2       Running   0          17m
node-exporter-7zx6s                    1/1       Running   0          17m
node-exporter-b88t5                    1/1       Running   0          17m
node-exporter-dhx7x                    1/1       Running   0          17m
node-exporter-wnhvd                    1/1       Running   0          17m
prometheus-k8s-0                       2/2       Running   0          16m
prometheus-k8s-1                       2/2       Running   0          16m
prometheus-operator-66578f9cd9-wstrp   1/1       Running   0          17m

[root@node-131 kube-prometheus]# kubectl get statefulset -n monitoring
NAME                DESIRED   CURRENT   AGE
alertmanager-main   3         3         16m
prometheus-k8s      2         2         16m


SecurityContextDeny
：具体的作用需查看官档。

a575

三、测试访问：
访问prometheus（30900）：



访问alertmanager（30903）：



访问grafana（30902）：







至此kube-prometheus框架基本搭建完成，后续关于prometheus具体操作另写：

本系列其他内容：

01-环境准备

02-etcd群集搭建

03-kubectl管理工具

04-master搭建

05-node节点搭建

06-addon-calico

07-addon-kubedns

08-addon-dashboard

09-addon-kube-prometheus

10-addon-EFK

11-addon-Harbor

12-addon-ingress-nginx

13-addon-traefik

参考文档：

https://github.com/coreos/prometheus-operator/tree/master/contrib/kube-prometheus