k8s实践16:kubernetes监测探针简单测试
1.两种探针
readiness probe(就绪探针)
监测容器是否就绪?只有pod里的容器就绪,kubelet才会认为pod处于就绪状态.
就绪探针的作用是控制哪些pod可以作为svc的后端,如果pod不是就绪状态,就把它从svc load balancer中移除.
liveness probe(存活探针)
监测容器是否存活?如果容器中的应用出现问题,liveness将检测到容器不健康会通知kubelet,kubelet重启该pod容器.
2.使用探针的三种方式
官网介绍了三种,见下:
command命令执行
http request访问
tcp socket连接
个人比较喜欢用第三种方式,tcp socket.
3.tcp socket方式学习测试
tcp socket方式
这个方式比较好理解.
比如说,起一个nginx容器,nginx服务提供的端口是80端口.
配置tcp socket 探针,设定隔一个时间,用tcp方式连接80端口,如果连接成功,就返回容器健康或者就绪,如果连接失败,返回容器不健康或者不就绪,kubelet重启容器.
逆向思维示例:
简单思路:探针tcp socket连接不存在的8080端口,必然连接失败报错,从而实现pod不断重启.
[root@k8s-master1 k8s_tanzhen]# cat tcp_ness apiVersion: v1 kind: Pod metadata: name: httpd labels: app: httpd spec: containers: - name: httpd image: nginx ports: - containerPort: 80 readinessProbe: tcpSocket: port: 8080 initialDelaySeconds: 45 periodSeconds: 20 livenessProbe: tcpSocket: port: 8080 initialDelaySeconds: 45 periodSeconds: 20 [root@k8s-master1 k8s_tanzhen]#
起一个nginx的pod容器,提供服务端口80.
配置探针连接端口8080,第一次监测时间为pod容器启动后的45s,第一次监测后每隔20s监测一次.
测试结果,pod容器一直在重启.
[root@k8s-master1 k8s_tanzhen]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE httpd 0/1 CrashLoopBackOff 7 18m 172.30.35.3 k8s-master3
describe报错
Warning Unhealthy 6m (x19 over 16m) kubelet, k8s-master3 Liveness probe failed: dial tcp 172.30.35.3:8080: connect: connection refused Warning Unhealthy 2m (x15 over 16m) kubelet, k8s-master3 Readiness probe failed: dial tcp 172.30.35.3:8080: connect: connection refused
探针自动tcp连接容器ip:8080端口,失败.所以容器一直重启.
正常配置示例
正常配置是连接提供服务的80端口
简单思路:理论上来说,长时间运行的应用程序最终会过渡到中断状态,除非重新启动,否则无法恢复.Kubernetes提供了活性探针来检测和补救这种情况.这是配置探针的根本原因,以防万一.
[root@k8s-master1 k8s_tanzhen]# cat tcp_ness apiVersion: v1 kind: Pod metadata: name: httpd labels: app: httpd spec: containers: - name: httpd image: nginx ports: - containerPort: 80 readinessProbe: tcpSocket: port: 80 initialDelaySeconds: 45 periodSeconds: 20 livenessProbe: tcpSocket: port: 80 initialDelaySeconds: 45 periodSeconds: 20
[root@k8s-master1 k8s_tanzhen]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE httpd 1/1 Running 0 2m 172.30.35.3 k8s-master3
正常配置模拟测试案例
简单思路:起nginx容器,然后执行命令杀死nginx进程,设定探针监测连接tcp socket 80端口,当nginx进程被杀死后,tcp socket连接失败,探针监测容器为不健康不就绪,kubelet重启容器.
[root@k8s-master1 k8s_tanzhen]# cat tcp_ness apiVersion: v1 kind: Pod metadata: name: httpd labels: app: httpd spec: containers: - name: httpd image: nginx args: - /bin/sh - -c - sleep 60;nginx -s quit ports: - containerPort: 80 readinessProbe: tcpSocket: port: 80 initialDelaySeconds: 20 periodSeconds: 10 livenessProbe: tcpSocket: port: 80 initialDelaySeconds: 20 periodSeconds: 10 [root@k8s-master1 k8s_tanzhen]#
配置参数说明:
容器启动后,执行nginx -s quit杀死Nginx进程
容器启动20s后开始执行readiness和liveness检测
容器启动后35s左右
探针监测到nginx进程已经死掉,无法连接到80端口,报警见下:
Warning Unhealthy 8s (x3 over 28s) kubelet, k8s-master3 Liveness probe failed: dial tcp 172.30.35.3:80: connect: connection refused Warning Unhealthy 7s (x3 over 27s) kubelet, k8s-master3 Readiness probe failed: dial tcp 172.30.35.3:80: connect: connection refused
整个重启事件记录
Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 2m default-scheduler Successfully assigned default/httpd to k8s-master3 Normal Pulled 1m (x2 over 2m) kubelet, k8s-master3 Successfully pulled image "nginx" Normal Created 1m (x2 over 2m) kubelet, k8s-master3 Created container Normal Started 1m (x2 over 2m) kubelet, k8s-master3 Started container Warning Unhealthy 16s (x6 over 1m) kubelet, k8s-master3 Liveness probe failed: dial tcp 172.30.35.3:80: connect: connection refed Warning Unhealthy 15s (x8 over 1m) kubelet, k8s-master3 Readiness probe failed: dial tcp 172.30.35.3:80: connect: connection resed Normal Pulling 5s (x3 over 2m) kubelet, k8s-master3 pulling image "nginx" Normal Killing 5s (x2 over 1m) kubelet, k8s-master3 Killing container with id docker://httpd:Container failed liveness prob. Container will be killed and recreated.
可以看到,nginx进程杀死后,pod自动重启.
[root@k8s-master1 k8s_tanzhen]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE httpd 0/1 Running 4 5m 172.30.35.3 k8s-master3
实现测试目的
- 点赞
- 收藏
- 分享
- 文章举报
- Kubernetes : 多节点 k8s 集群搭建实践
- 北航课程 软件测试工具与实践16: IBM Rational Product Overview(Optional)
- kubernetes实战之consul篇及consul在windows下搭建consul简单测试环境
- 01 | 健康之路 kubernetes(k8s) 实践之路 : 开篇及概况
- kubernetes(k8s)搭建实践
- k8s的容器监测探针
- DCOS之k8s的容器监测探针
- kubernetes(k8s)以及涉及组件简单介绍
- Linux编程实践——文件I/O缓冲区测试及cat简单实现
- Json实践(一):fastjson和gson简单效率测试
- 如何简单的测试kubernetes的dns add-ons是否工作正常?
- 关于等价类测试的简单实践 20150322
- k8s实践19:kubernetes二进制部署集群v1.12升级v1.15
- Docker+Kubernetes(k8s)微服务容器化实践视频课程
- Jmeter简单介绍与搭配Jenkins实现自动化测试实践
- JNotify的监测文件变化的简单测试例子
- aspnetcore.webapi实践k8s健康探测机制 - kubernetes
- Kubernetes(K8s)容器设计模式实践案例 单节点多容器模式
- k8s实践17:kubernetes对接nfs存储实现pvc动态按需创建分配绑定pv
- 关于等价类测试的简单实践(二) 20150329