腾讯云tke容器服务调整kubelet工作目录
2019-12-23 17:54
916 查看
腾讯云默认kubelet工作目录在/var/lib/kubelet使用的是系统盘,系统盘一般不够大也不方便扩容,所以需要修改kubelet工作目录。大概流程如下:
-
准备工作
-
驱逐节点
-
修改配置
-
重启服务
-
解封节点
- 验证服务
准备工作
查找kubelet启动文件,一般配置信息在启动文件中
[root@VM_8_9_centos ~]# locate kubelet.service /etc/systemd/system/multi-user.target.wants/kubelet.service /usr/lib/systemd/system/kubelet.service [root@VM_8_9_centos ~]# ls -lh /usr/lib/systemd/system/kubelet.service [root@VM_8_9_centos ~]# ls -lh /etc/systemd/system/multi-user.target.wants/kubelet.service lrwxrwxrwx 1 root root 39 Sep 25 11:59 /etc/systemd/system/multi-user.target.wants/kubelet.service -> /usr/lib/systemd/system/kubelet.service [root@VM_8_9_centos ~]# cat /usr/lib/systemd/system/kubelet.service [Unit] Description=kubelet [Service] Environment=QCLOUD_NORM_URL= EnvironmentFile=-/etc/kubernetes/kubelet ExecStart=/usr/bin/kubelet ${CNI_BIN_DIR} ${KUBE_RESERVED} ${MAX_PODS} ${CLOUD_PROVIDER} ${CLOUD_CONFIG} ${POD_INFRA_CONTAINER_IMAGE} ${CLUSTER_DOMAIN} ${ALLOW_PRIVILEGED} ${REGISTER_SCHEDULABLE} ${FAIL_SWAP_ON} ${ANONYMOUS_AUTH} ${IMAGE_PULL_PROGRESS_DEADLINE} ${HOSTNAME_OVERRIDE} ${EVICTION_HARD} ${AUTHENTICATION_TOKEN_WEBHOOK} ${CLIENT_CA_FILE} ${AUTHORIZATION_MODE} ${CLUSTER_DNS} ${NON_MASQUERADE_CIDR} ${NETWORK_PLUGIN} ${KUBECONFIG} ${V} ExecStartPost=-/bin/bash /etc/kubernetes/deny-tcp-port-10250.sh Restart=always RestartSec=10 LimitNOFILE=65536 [Install] WantedBy=multi-user.target
根据启动文件发现环境文件在kubelet中
[root@VM_8_9_centos ~]# cat /etc/kubernetes/kubelet CNI_BIN_DIR="--cni-bin-dir=/opt/cni/bin/" KUBE_RESERVED="--kube-reserved=cpu=90m,memory=1830Mi" MAX_PODS="--max-pods=253" CLOUD_PROVIDER="--cloud-provider=qcloud" CLOUD_CONFIG="--cloud-config=/etc/kubernetes/qcloud.conf" POD_INFRA_CONTAINER_IMAGE="--pod-infra-container-image=ccr.ccs.tencentyun.com/library/pause:latest" CLUSTER_DOMAIN="--cluster-domain=cluster.local" ALLOW_PRIVILEGED="--allow-privileged=true" REGISTER_SCHEDULABLE="--register-schedulable=true" FAIL_SWAP_ON="--fail-swap-on=false" ANONYMOUS_AUTH="--anonymous-auth=false" IMAGE_PULL_PROGRESS_DEADLINE="--image-pull-progress-deadline=10m0s" HOSTNAME_OVERRIDE="--hostname-override=192.168.8.9" EVICTION_HARD="--eviction-hard=nodefs.available<10%,nodefs.inodesFree<5%,imagefs.available<15%,memory.available<100Mi" AUTHENTICATION_TOKEN_WEBHOOK="--authentication-token-webhook=true" CLIENT_CA_FILE="--client-ca-file=/etc/kubernetes/cluster-ca.crt" AUTHORIZATION_MODE="--authorization-mode=Webhook" CLUSTER_DNS="--cluster-dns=172.16.255.48" NON_MASQUERADE_CIDR="--non-masquerade-cidr=0.0.0.0/0" NETWORK_PLUGIN="--network-plugin=cni" KUBECONFIG="--kubeconfig=/etc/kubernetes/kubelet-kubeconfig" V="--v=2"
驱逐节点
驱逐会自动封锁节点(将节点内pod驱逐到集群中其他节点上,不影响集群使用,DaemonSet pod除外),驱逐后会检查节点为不可调度状态
[root@VM_8_9_centos ~]# kubectl drain 192.168.8.9 node/192.168.8.9 cordoned error: unable to drain node "192.168.8.9", aborting command... There are pending nodes to be drained: 192.168.8.9 error: DaemonSet-managed pods (use --ignore-daemonsets to ignore): rancher-logging-fluentd-tlqnq, rancher-logging-log-aggregator-m5g69, registry-proxy-xkmk5, exporter-node-cluster-monitoring-t5jj7, cattle-node-agent-j8wcn, ccs-log-collector-x57f8, ip-masq-agent-xt9q2, tke-bridge-agent-8rxgl, tke-cni-agent-lvv98; pods with local storage (use --delete-local-data to override): coupon-service-1, yuedi-passenger-api-1, coupon-service-1, rental-api-1, passenger-api-0, eureka-1, service-9777bd6ff-bsnj7, rental-service-0, passenger-api-0, swift-778d866966-c6nk7 [root@VM_8_9_centos ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION 192.168.8.14 Ready <none> 89d v1.12.4-tke.10 192.168.8.15 Ready <none> 181d v1.12.4-tke.3 192.168.8.3 Ready <none> 181d v1.12.4-tke.3 192.168.8.4 Ready <none> 181d v1.12.4-tke.3 192.168.8.9 Ready,SchedulingDisabled <none> 89d v1.12.4-tke.10 [root@VM_8_9_centos ~]# ls /data/ docker kubelet lost+found
修改配置
修改kubelet配置文件第一行添加kubelet 目录参数
[root@VM_8_9_centos ~]# head -n 1 /etc/kubernetes/kubelet ROOT_DIR="--root-dir=/data/kubelet"
修改启动配置文件添加ROOT_DIR参数
[root@VM_8_9_centos ~]# vim /usr/lib/systemd/system/kubelet.service ExecStart=/usr/bin/kubelet ${ROOT_DIR} ${CNI_BIN_DIR} ${KUBE_RESERVED} ${MAX_PODS} ${CLOUD_PROVIDER} ${CLOUD_CONFIG} ${POD_INFRA_CONTAINER_IMAGE} ${CLUSTER_DOMAIN} ${ALLOW_PRIVILEGED} ${REGISTER_SCHEDULABLE} ${FAIL_SWAP_ON} ${ANONYMOUS_AUTH} ${IMAGE_PULL_PROGRESS_DEADLINE} ${HOSTNAME_OVERRIDE} ${EVICTION_HARD} ${AUTHENTICATION_TOKEN_WEBHOOK} ${CLIENT_CA_FILE} ${AUTHORIZATION_MODE} ${CLUSTER_DNS} ${NON_MASQUERADE_CIDR} ${NETWORK_PLUGIN} ${KUBECONFIG} ${V}
重启服务
[root@VM_8_9_centos ~]# systemctl daemon-reload [root@VM_8_9_centos ~]# systemctl restart kubelet [root@VM_8_9_centos ~]# systemctl status kubelet -l ● kubelet.service - kubelet Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)ls / Active: active (running) since Mon 2019-12-23 16:38:04 CST; 1min 2s ago Process: 24745 ExecStartPost=/bin/bash /etc/kubernetes/deny-tcp-port-10250.sh (code=exited, status=0/SUCCESS) Main PID: 24744 (kubelet) Tasks: 26 Memory: 49.2M CGroup: /system.slice/kubelet.service └─24744 /usr/bin/kubelet --root-dir=/data/kubelet --cni-bin-dir=/opt/cni/bin/ --kube-reserved=cpu=90m,memory=1830Mi --max-pods=253 --cloud-provider=qcloud --cloud-config=/etc/kubernetes/qcloud.conf --pod-infra-container-image=ccr.ccs.tencentyun.com/library/pause:latest --cluster-domain=cluster.local --allow-privileged=true --register-schedulable=true --fail-swap-on=false --anonymous-auth=false --image-pull-progress-deadline=10m0s --hostname-override=192.168.8.9 --eviction-hard=nodefs.available<10%,nodefs.inodesFree<5%,imagefs.available<15%,memory.available<100Mi --authentication-token-webhook=true --client-ca-file=/etc/kubernetes/cluster-ca.crt --authorization-mode=Webhook --cluster-dns=172.16.255.48 --non-masquerade-cidr=0.0.0.0/0 --network-plugin=cni --kubeconfig=/etc/kubernetes/kubelet-kubeconfig --v=2
取消封锁
[root@VM_8_9_centos ~]# kubectl uncordon 192.168.8.9 node/192.168.8.9 uncordoned
验证效果
查看节点信息正常
[root@VM_8_9_centos ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION 192.168.8.14 Ready <none> 89d v1.12.4-tke.10 192.168.8.15 Ready <none> 181d v1.12.4-tke.3 192.168.8.3 Ready <none> 181d v1.12.4-tke.3 192.168.8.4 Ready <none> 181d v1.12.4-tke.3 192.168.8.9 Ready <none> 89d v1.12.4-tke.10
新建kubelet目录在data下
[root@VM_8_9_centos ~]# ls /data/kubelet/ cpu_manager_state plugin-containers plugins pods [root@VM_8_9_centos ~]# du -sh /data/kubelet/ 480K /data/kubelet/
相关文章推荐
- docker Win8.1 docker toolbox tomcat 将本机工作目录挂载到 tomcat 容器的工作目录
- 腾讯云容器服务大容量日志的处理记录
- 如果修改Windows服务应用程序工作目录?
- .Net Core 微服务容器系列基础目录篇
- tomcat容器中使用JNDI访问外部openLDAP提供的目录服务
- 无法在当前的 Active Directory 域中创建服务连接点。验证 SharePoint 容器是否存在于当前域中并验证您是否具有向其写入的权限。 Microsoft.SharePoint.SPException: 目录中不存在对象 LDAP://CN=Microsoft SharePoint
- codis构建docker镜像然后发布到阿里云容器服务前期准备工作
- NFS奇怪问题1 - NFS无法工作,服务重启失败,挂载目录变成问号。
- 腾讯云容器服务的滚动升级使用简介
- PHPStorm更改为Apache服务端口,及修改默认的网站目录为PHPStorm的工作目录
- 腾讯云容器服务的滚动升级使用简介
- 使用java实现腾讯云存储服务(COSClient)
- PHP解耦的三重境界(浅谈服务容器)
- 重设“目录服务还原模式”的系统管理员密码
- 使用腾讯云提供的针对Nuget包管理器的缓存加速服务
- 在服务程序中,获取当前用户目录(如:C:\Documents and Settings\The God\Application Data)
- 【收藏】C#中得到程序当前工作目录和执行目录的一些方法
- 如何启动eclipse时提示选择工作空间||删除workspace空间的目录
- 总结C#中得到程序当前工作目录和执行目录的一些方法
- Opt目录满导致数据库ons服务OFFLINE