您的位置:首页 > 其它

k8s实践19:kubernetes二进制部署集群v1.12升级v1.15

2020-02-02 00:28 1056 查看

1.
升级前的版本

[root@k8s-master1 ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.4", GitCommit:"5ca598b4ba5abb89bb773071ce452e33fb66339d", GitTreeState:"clean", BuildDate:"2018-06-06T08:13:03Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.3", GitCommit:"435f92c719f279a3a67808c80521ea17d5715c66", GitTreeState:"clean", BuildDate:"2018-11-26T12:46:57Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}

一个疑问:
kubectl version获取的client版本和server版本数据来自哪里呢?
client版本是kubectl执行文件的版本.
server版本是kube-apiserver执行文件的版本.

如果把kubectl的配置文件,~/.kube/config文件移走,无法识别server地址,读取不到server版本参数.报错见下:

[root@k8s-master1 .kube]# kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.4", GitCommit:"5ca598b4ba5abb89bb773071ce452e33fb66339d", GitTreeState:"clean", BuildDate:"2018-06-06T08:13:03Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Error from server (NotFound): the server could not find the requested resource

2.
细节准备

集群三个节点,三个节点都是master节点同时也是node节点.
二进制部署,配置文件不变,替换执行文件.
需要升级的组件:

master节点组件:
kubectl
kube-apiserver
kube-controller-manager
kube-scheduler

node节点组件:
kube-proxy
kubelet

暂不升级的组件,需要升级后面再单独升级
数据库组件:
etcd
网络组件:
flannel

升级前,执行文件的存放路径和权限检索记录:

[root@k8s-master1 bin]# pwd
/opt/k8s/bin
[root@k8s-master1 bin]# ll
total 819520
-rwxr-xr-x 1 k8s  root  10376657 Feb 26 13:27 cfssl
-rwxr-xr-x 1 k8s  root  6595195 Feb 26 13:27 cfssl-certinfo
-rwxr-xr-x 1 k8s  root  2277873 Feb 26 13:27 cfssljson
-rwxr-xr-x 1 k8s  root      1752 Feb 26 13:21 environment.sh
-rwxr-xr-x 1 k8s  root  19266976 Feb 26 14:10 etcd
-rwxr-xr-x 1 k8s  root  16018720 Feb 26 14:10 etcdctl
-rwxr-xr-x 1 k8s  root  36327752 Feb 26 15:43 flanneld
-rwxr-xr-x 1 root root  54038482 Feb 28 10:17 kubeadm
-rwxr-xr-x 1 k8s  root 192793815 Feb 27 14:40 kube-apiserver
-rwxr-xr-x 1 k8s  root 162973612 Feb 27 14:40 kube-controller-manager
-rwxr-xr-x 1 k8s  root  54308597 Feb 26 13:39 kubectl
-rwxr-xr-x 1 root root 176661512 Feb 28 10:17 kubelet
-rwxr-xr-x 1 root root  50330867 Feb 28 10:17 kube-proxy
-rwxr-xr-x 1 k8s  root  57184656 Feb 27 14:40 kube-scheduler
-rwxr-xr-x 1 k8s  root      2139 Feb 26 15:43 mk-docker-opts.sh
[root@k8s-master1 bin]#
[root@k8s-master1 bin]# echo $PATH
/opt/k8s/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:/root/bin
[root@k8s-master1 bin]#
[root@k8s-master1 ~]# cat .bashrc
# .bashrc

# User specific aliases and functions

alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'

# Source global definitions
if [ -f /etc/bashrc ]; then
        . /etc/bashrc
fi
export PATH=/opt/k8s/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
[root@k8s-master1 ~]#

3.
升级文件

下载地址参考:
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.15.md

下载的文件,参考见下:

[root@k8s-master1 v1.15.0]# pwd
/root/k8s/v1.15.0
[root@k8s-master1 v1.15.0]# ls
 kubernetes-client-linux-amd64.tar.gz  kubernetes-server-linux-amd64.tar.gz  kubernetes-src.tar.gz  kubernetes.tar.gz
[root@k8s-master1 v1.15.0]#

一般只需要

 kubernetes-client-linux-amd64.tar.gz  kubernetes-server-linux-amd64.tar.gz

4.
升级最简单的kubectl

先备份kubectl执行文件,如果有问题可以回滚.三个节点都一样操作.

[root@k8s-master1 bin]# pwd
/opt/k8s/bin
[root@k8s-master1 bin]# cp kubectl kubectl.bk.v1.12
[root@k8s-master1 bin]# ll
total 872556
-rwxr-xr-x 1 k8s  root  10376657 Feb 26 13:27 cfssl
-rwxr-xr-x 1 k8s  root  6595195 Feb 26 13:27 cfssl-certinfo
-rwxr-xr-x 1 k8s  root  2277873 Feb 26 13:27 cfssljson
-rwxr-xr-x 1 k8s  root      1752 Feb 26 13:21 environment.sh
-rwxr-xr-x 1 k8s  root  19266976 Feb 26 14:10 etcd
-rwxr-xr-x 1 k8s  root  16018720 Feb 26 14:10 etcdctl
-rwxr-xr-x 1 k8s  root  36327752 Feb 26 15:43 flanneld
-rwxr-xr-x 1 root root  54038482 Feb 28 10:17 kubeadm
-rwxr-xr-x 1 k8s  root 192793815 Feb 27 14:40 kube-apiserver
-rwxr-xr-x 1 k8s  root 162973612 Feb 27 14:40 kube-controller-manager
-rwxr-xr-x 1 k8s  root  54308597 Feb 26 13:39 kubectl
-rwxr-xr-x 1 root root  54308597 Jul 30 15:10 kubectl.bk.v1.12
-rwxr-xr-x 1 root root 176661512 Feb 28 10:17 kubelet
-rwxr-xr-x 1 root root  50330867 Feb 28 10:17 kube-proxy
-rwxr-xr-x 1 k8s  root  57184656 Feb 27 14:40 kube-scheduler
-rwxr-xr-x 1 k8s  root      2139 Feb 26 15:43 mk-docker-opts.sh

删除kubectl文件

[root@k8s-master1 bin]# rm -rf kubectl

复制新版本的kubectl到目录/opt/k8s/bin

[root@k8s-master1 v1.15.0]# pwd
/root/k8s/v1.15.0
[root@k8s-master1 v1.15.0]# cp kubernetes/client/bin/kubectl /opt/k8s/bin
[root@k8s-master1 v1.15.0]# scp kubernetes/client/bin/kubectl k8s-master2:/opt/k8s/bin
kubectl                                                                                                    100%  41MB  10.3MB/s  00:04   
[root@k8s-master1 v1.15.0]# scp kubernetes/client/bin/kubectl k8s-master3:/opt/k8s/bin
kubectl       

执行命令看看

[root@k8s-master1 v1.15.0]# kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:40:16Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.3", GitCommit:"435f92c719f279a3a67808c80521ea17d5715c66", GitTreeState:"clean", BuildDate:"2018-11-26T12:46:57Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}

可以看到kubectl client已经升级到v1.15.0

5.
升级
kube-apiserver
kube-controller-manager
kube-scheduler

5.1
这里是测试环境,随便搞都可以,如果是正式环境,慎重操作,最好一个一个master节点去升级操作.
备份操作忽略,如果没有旧版本安装包的存档,请自行备份.

停止kube-scheduler,kube-controller-manager,kube-apiserver服务,三节点同时操作:

[root@k8s-master1 bin]# systemctl stop kube-scheduler
[root@k8s-master2 bin]# systemctl stop kube-scheduler
[root@k8s-master3 bin]# systemctl stop kube-scheduler

[root@k8s-master1 bin]# systemctl stop kube-controller-manager
[root@k8s-master2 bin]# systemctl stop kube-controller-manager
[root@k8s-master3 bin]# systemctl stop kube-controller-manager

[root@k8s-master1 bin]# systemctl stop kube-apiserver
[root@k8s-master2 bin]# systemctl stop kube-apiserver
[root@k8s-master3 bin]# systemctl stop kube-apiserver

5.2
删除旧版本执行文件

[root@k8s-master1 bin]# rm -rf kube-apiserver kube-scheduler kube-controller-manager
[root@k8s-master2 bin]# rm -rf kube-apiserver kube-scheduler kube-controller-manager
[root@k8s-master3 bin]# rm -rf kube-apiserver kube-scheduler kube-controller-manager

5.3
复制新版本执行文件到/opt/k8s/bin目录

[root@k8s-master1 bin]# cp kube-apiserver kube-controller-manager kube-scheduler /opt/k8s/bin
[root@k8s-master1 bin]# scp kube-apiserver kube-controller-manager kube-scheduler k8s-master2:/opt/k8s/bin
kube-apiserver                                                                                          100%  157MB  9.8MB/s  00:16   
kube-controller-manager                                                                                  100%  111MB  12.3MB/s  00:09   
kube-scheduler                                                                                          100%  37MB  9.3MB/s  00:04   
[root@k8s-master1 bin]# scp kube-apiserver kube-controller-manager kube-scheduler k8s-master3:/opt/k8s/bin
kube-apiserver                                                                                          100%  157MB  9.8MB/s  00:16   
kube-controller-manager                                                                                  100%  111MB  12.3MB/s  00:09   
kube-scheduler                                                                                          100%  37MB  12.3MB/s  00:03   
[root@k8s-master1 bin]# pwd
/root/k8s/v1.15.0/kubernetes/server/bin
[root@k8s-master1 bin]#

5.4
修改权限,参考命令

chown -R k8s:root /opt/k8s/bin

5.5
遇到的报错

启动服务
kube-apiserver起不来,报错:

Jul 30 16:04:47 k8s-master1 kube-apiserver[108963]: error: enable-admission-plugins plugin "Initializers" is unknown

[root@k8s-master1 kubernetes]# cat /etc/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target
[Service]
ExecStart=/opt/k8s/bin/kube-apiserver \
--enable-admission-plugins=Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota \
--anonymous-auth=false \
--experimental-encryption-provider-config=/etc/kubernetes/encryption-config.yaml \
--advertise-address=192.168.32.128 \
--bind-address=192.168.32.128 \
--insecure-port=0 \
--authorization-mode=Node,RBAC \
--runtime-config=api/all \
--enable-bootstrap-token-auth \
--service-cluster-ip-range=10.254.0.0/16 \
--service-node-port-range=8400-9000 \
--tls-cert-file=/etc/kubernetes/cert/kubernetes.pem \
--tls-private-key-file=/etc/kubernetes/cert/kubernetes-key.pem \
--client-ca-file=/etc/kubernetes/cert/ca.pem \
--kubelet-client-certificate=/etc/kubernetes/cert/kubernetes.pem \
--kubelet-client-key=/etc/kubernetes/cert/kubernetes-key.pem \
--service-account-key-file=/etc/kubernetes/cert/ca-key.pem \
--etcd-cafile=/etc/kubernetes/cert/ca.pem \
--etcd-certfile=/etc/kubernetes/cert/kubernetes.pem \
--etcd-keyfile=/etc/kubernetes/cert/kubernetes-key.pem \
--etcd-servers=https://192.168.32.128:2379,https://192.168.32.129:2379,https://192.168.32.130:2379 \
--enable-swagger-ui=true \
--allow-privileged=true \
--apiserver-count=3 \
--audit-log-maxage=30 \
--audit-log-maxbackup=3 \
--audit-log-maxsize=100 \
--audit-log-path=/var/log/kube-apiserver-audit.log \
--event-ttl=1h \
--alsologtostderr=true \
--logtostderr=false \
--log-dir=/var/log/kubernetes \
--v=2
Restart=on-failure
RestartSec=5
Type=notify
User=k8s
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
[root@k8s-master1 kubernetes]#
--enable-admission-plugins=Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota

删除Initializers,再启动正常.

[root@k8s-master1 kubernetes]# systemctl daemon-reload &&systemctl start kube-apiserver
● kube-apiserver.service - Kubernetes API Server
  Loaded: loaded (/etc/systemd/system/kube-apiserver.service; enabled; vendor preset: disabled)
  Active: active (running) since Tue 2019-07-30 16:09:35 CST; 2min 38s ago
    Docs: https://github.com/GoogleCloudPlatform/kubernetes
Main PID: 110572 (kube-apiserver)
  Memory: 309.6M
  CGroup: /system.slice/kube-apiserver.service
          └─110572 /opt/k8s/bin/kube-apiserver --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota --anonymous-auth=false --experimental-encryption-provider-config=/etc/kubernetes/encryption-config.yaml --advertise-address=192.168.32.128 --bind-address=192.168.32.128 --insecure-port=0 --authorization-mode=Node,RBAC --runtime-config=api/all --enable-bootstrap-token-auth --service-cluster-ip-range=10.254.0.0/16 --service-node-port-range=8400-9000 --tls-cert-file=/etc/kubernetes/cert/kubernetes.pem --tls-private-key-file=/etc/kubernetes/cert/kubernetes-key.pem --client-ca-file=/etc/kubernetes/cert/ca.pem --kubelet-client-certificate=/etc/kubernetes/cert/kubernetes.pem --kubelet-client-key=/etc/kubernetes/cert/kubernetes-key.pem --service-account-key-file=/etc/kubernetes/cert/ca-key.pem --etcd-cafile=/etc/kubernetes/cert/ca.pem --etcd-certfile=/etc/kubernetes/cert/kubernetes.pem --etcd-keyfile=/etc/kubernetes/cert/kubernetes-key.pem --etcd-servers=https://192.168.32.128:2379,https://192.168.32.129:2379,https://192.168.32.130:2379 --enable-swagger-ui=true --allow-privileged=true --apiserver-count=3 --audit-log-maxage=30 --audit-log-maxbackup=3 --audit-log-maxsize=100 --audit-log-path=/var/log/kube-apiserver-audit.log --event-ttl=1h --alsologtostderr=true --logtostderr=false --log-dir=/var/log/kubernetes --v=2

其他两个节点也这样操作即可.

5.6
启动scheduler和controller-manager服务,参考命令:

systemctl start kube-controller-manager &&systemctl start kube-scheduler

5.7
执行命令检索版本

[root@k8s-master1 kubernetes]# kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:40:16Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:32:14Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

6.
升级
kube-proxy
kubelet

6.1停止服务

[root@k8s-master1 kubernetes]# systemctl stop kubelet && systemctl stop kube-proxy
[root@k8s-master2 bin]# systemctl stop kubelet && systemctl stop kube-proxy
[root@k8s-master3 bin]# systemctl stop kubelet && systemctl stop kube-proxy

6.2
删除旧版本执行文件

[root@k8s-master1 bin]# rm -rf kubelet kube-proxy
[root@k8s-master2 bin]#  rm -rf kubelet kube-proxy
[root@k8s-master3 bin]#  rm -rf kubelet kube-proxy
[root@k8s-master3 bin]# pwd
/opt/k8s/bin

6.3
复制新版本执行文件到目录

[root@k8s-master1 bin]# pwd
/root/k8s/v1.15.0/kubernetes/server/bin
[root@k8s-master1 bin]# cp kubelet kube-proxy /opt/k8s/bin
[root@k8s-master1 bin]# scp kubelet kube-proxy k8s-master2:/opt/k8s/bin
kubelet                                                                                                              100%  114MB  11.4MB/s  00:10   
kube-proxy                                                                                                          100%  35MB  11.8MB/s  00:03   
[root@k8s-master1 bin]# scp kubelet kube-proxy k8s-master3:/opt/k8s/bin
kubelet                                                                                                              100%  114MB  12.7MB/s  00:09   
kube-proxy 

6.4
修改下权限

[root@k8s-master1 bin]# chown -R k8s:root /opt/k8s/bin
[root@k8s-master1 bin]# ssh root@k8s-master2 "chown -R k8s:root /opt/k8s/bin"
[root@k8s-master1 bin]# ssh root@k8s-master3 "chown -R k8s:root /opt/k8s/bin"

 6.5
启动服务
kube-proxy启动正常

6.6
kubelet服务启动不成功

[root@k8s-master1 bin]# systemctl status kubelet -l
● kubelet.service - Kubernetes Kubelet
  Loaded: loaded (/etc/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Active: activating (auto-restart) (Result: exit-code) since Tue 2019-07-30 16:53:57 CST; 1s ago
    Docs: https://github.com/GoogleCloudPlatform/kubernetes
  Process: 122129 ExecStart=/opt/k8s/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/kubelet-bootstrap.kubeconfig --cert-dir=/etc/kubernetes/cert --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --config=/etc/kubernetes/kubelet.config.json --hostname-override=k8s-master1 --pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest --allow-privileged=true --alsologtostderr=true --logtostderr=false --log-dir=/var/log/kubernetes --v=2 (code=exited, status=255)
Main PID: 122129 (code=exited, status=255)

Jul 30 16:53:57 k8s-master1 kubelet[122129]: --tls-cert-file string                                                                                      File containing x509 Certificate used for serving HTTPS (with intermediate certs, if any, concatenated after server cert). If --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory passed to --cert-dir. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jul 30 16:53:57 k8s-master1 kubelet[122129]: --tls-cipher-suites strings                                                                                Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used. Possible values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_RC4_128_SHA,TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_RC4_128_SHA (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jul 30 16:53:57 k8s-master1 kubelet[122129]: --tls-min-version string                                                                                    Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13 (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jul 30 16:53:57 k8s-master1 kubelet[122129]: --tls-private-key-file string                                                                              File containing x509 private key matching --tls-cert-file. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jul 30 16:53:57 k8s-master1 kubelet[122129]: -v, --v Level                                                                                                  number for the log level verbosity
Jul 30 16:53:57 k8s-master1 kubelet[122129]: --version version[=true]                                                                                    Print version information and quit
Jul 30 16:53:57 k8s-master1 kubelet[122129]: --vmodule moduleSpec                                                                                        comma-separated list of pattern=N settings for file-filtered logging
Jul 30 16:53:57 k8s-master1 kubelet[122129]: --volume-plugin-dir string                                                                                  The full path of the directory in which to search for additional third party volume plugins (default "/usr/libexec/kubernetes/kubelet-plugins/volume/exec/")
Jul 30 16:53:57 k8s-master1 kubelet[122129]: --volume-stats-agg-period duration                                                                          Specifies interval for kubelet to calculate and cache the volume disk usage for all pods and volumes.  To disable volume calculations, set to 0. (default 1m0s) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jul 30 16:53:57 k8s-master1 kubelet[122129]: F0730 16:53:57.422346  122129 server.go:156] unknown flag: --allow-privileged
[root@k8s-master1 bin]#

这是一个bug报错:
参考下面解决两篇文章解决的
https://github.com/microsoft/SDN/issues/379
https://github.com/wk8/SDN/commit/8db8f06cd5ccdc91eb74ce1d00041597881cd0c1

[root@k8s-master1 bin]# cat /etc/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service

[Service]
WorkingDirectory=/var/lib/kubelet
ExecStart=/opt/k8s/bin/kubelet \
  --bootstrap-kubeconfig=/etc/kubernetes/kubelet-bootstrap.kubeconfig \
  --cert-dir=/etc/kubernetes/cert \
  --kubeconfig=/etc/kubernetes/kubelet.kubeconfig \
  --config=/etc/kubernetes/kubelet.config.json \
  --hostname-override=k8s-master1 \
  --pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest \
  --allow-privileged=true \
  --alsologtostderr=true \
  --logtostderr=false \
  --log-dir=/var/log/kubernetes \
  --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

把  --allow-privileged=true \这个参数删除后,启动正常.

[root@k8s-master1 bin]# systemctl daemon-reload &&systemctl restart kubelet
[root@k8s-master1 bin]# systemctl status kubelet
● kubelet.service - Kubernetes Kubelet
  Loaded: loaded (/etc/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Active: active (running) since Tue 2019-07-30 17:11:19 CST; 9s ago
    Docs: https://github.com/GoogleCloudPlatform/kubernetes
Main PID: 127177 (kubelet)
  Memory: 61.0M
  CGroup: /system.slice/kubelet.service
          └─127177 /opt/k8s/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/kubelet-bootstrap.kubeconfig --cert-dir=/etc/kubernetes/cert --kubec...

Jul 30 17:11:20 k8s-master1 kubelet[127177]: I0730 17:11:20.658288  127177 state_mem.go:84] [cpumanager] updated default cpuset: ""
Jul 30 17:11:20 k8s-master1 kubelet[127177]: I0730 17:11:20.658314  127177 state_mem.go:92] [cpumanager] updated cpuset assignments: "map[]"
Jul 30 17:11:20 k8s-master1 kubelet[127177]: I0730 17:11:20.658337  127177 state_checkpoint.go:100] [cpumanager] state checkpoint: restored ...eckpoint
Jul 30 17:11:20 k8s-master1 kubelet[127177]: I0730 17:11:20.658345  127177 state_checkpoint.go:101] [cpumanager] state checkpoint: defaultCPUSet:
Jul 30 17:11:20 k8s-master1 kubelet[127177]: I0730 17:11:20.658427  127177 server.go:1025] Using root directory: /var/lib/kubelet
Jul 30 17:11:20 k8s-master1 kubelet[127177]: I0730 17:11:20.658452  127177 kubelet.go:306] Watching apiserver
Jul 30 17:11:20 k8s-master1 kubelet[127177]: I0730 17:11:20.688516  127177 client.go:75] Connecting to docker on unix:///var/run/docker.sock
Jul 30 17:11:20 k8s-master1 kubelet[127177]: I0730 17:11:20.688542  127177 client.go:104] Start docker client with request timeout=2m0s
Jul 30 17:11:20 k8s-master1 kubelet[127177]: W0730 17:11:20.751096  127177 docker_service.go:561] Hairpin mode set to "promiscuous-bridge" b...in-veth"
Jul 30 17:11:20 k8s-master1 kubelet[127177]: I0730 17:11:20.753080  127177 docker_service.go:238] Hairpin mode set to "hairpin-veth"
Hint: Some lines were ellipsized, use -l to show in full.

7.
升级完成

[root@k8s-master1 ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:40:16Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:32:14Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
[root@k8s-master1 ~]# kubectl cluster-info
Kubernetes master is running at https://192.168.32.127:8443
CoreDNS is running at https://192.168.32.127:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
[root@k8s-master1 ~]# kubectl get cs
NAME                 STATUS    MESSAGE             ERROR
controller-manager   Healthy   ok
scheduler            Healthy   ok
etcd-0               Healthy   {"health":"true"}
etcd-2               Healthy   {"health":"true"}
etcd-1               Healthy   {"health":"true"}
[root@k8s-master1 ~]# kubectl get nodes
NAME          STATUS   ROLES    AGE   VERSION
k8s-master1   Ready    <none>   98d   v1.15.0
k8s-master2   Ready    <none>   98d   v1.15.0
k8s-master3   Ready    <none>   98d   v1.15.0
[root@k8s-master1 ~]#
  • 点赞
  • 收藏
  • 分享
  • 文章举报
chadansheng4806 发布了0 篇原创文章 · 获赞 0 · 访问量 418 私信 关注
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: