离线使用 kubeadm 创建 kubernetes 1.9.2 集群

简介

kubeadm是一个kubernetes官方提供的快速安装和初始化拥有最佳实践的kubernetes集群的工具，但是目前不建议生产使用。

系统版本

OS：CentOS Linux release 7.3.1611 (Core)

Docker ：1.12.6

所需软件

链接：https://pan.baidu.com/s/1dzQyiq 密码：dyvi

初始化环境（所有节点上面执行）

关闭selinux

sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config

setenforce 0

关闭sawp分区

swapoff -a

vi /etc/fstab

注释掉swap分区

#/dev/mapper/cl-swap     swap                    swap    defaults        0 0

关闭系统防火墙

systemctl disable iptables-services firewalld
systemctl stop iptables-services firewalld

安装第epel yum源

yum install epel-release

安装docker

注意，kubead目前支持docker版本为docker-ce 17.4，这里使用yum自带的dcoekr版本

yum install -y docker

systemctl enable docker && sudo systemctl start docker

相关参数设置

cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sudo sysctl --system

安装kubeadm, kubelet和kubectl

需要文件
kubelet-1.9.2-0.x86_64.rpm
kubectl-1.9.2-0.x86_64.rpm
kubeadm-1.9.2-0.x86_64.rpm
kubernetes-cni-0.6.0-0.x86_64.rpm

yum localinstall  kubelet-1.9.2-0.x86_64.rpm kubectl-1.9.2-0.x86_64.rpm kubeadm-1.9.2-0.x86_64.rpm kubernetes-cni-0.6.0-0.x86_64.rpm -y

systemctl enable kubelet &&  systemctl start kubelet

安装Master

此操作只在Master节点执行

导入所需要镜像

需要文件
etcd-amd64.tar
flannel.tar
k8s-dns-dnsmasq-nanny-amd64.tar
k8s-dns-kube-dns-amd64.tar
k8s-dns-sidecar-amd64.tar
kube-apiserver-amd64.tar
kube-controller-manager-amd64.tar
kube-proxy-amd64.tar
kube-scheduler-amd64.tar
pause-amd64.tar
kube-flannel.ymal

for i in `ls`;do docker load < $i ;done

[root@master1]# docker images
REPOSITORY                                               TAG                 IMAGE ID            CREATED             SIZE
gcr.io/google_containers/kube-proxy-amd64                v1.9.2              e6754bb0a529        12 days ago         109.1 MB
gcr.io/google_containers/kube-controller-manager-amd64   v1.9.2              769d889083b6        12 days ago         137.8 MB
gcr.io/google_containers/kube-apiserver-amd64            v1.9.2              7109112be2c7        12 days ago         210.4 MB
gcr.io/google_containers/kube-scheduler-amd64            v1.9.2              2bf081517538        12 days ago         62.71 MB
gcr.io/google_containers/etcd-amd64                      3.1.11              59d36f27cceb        8 weeks ago         193.9 MB
quay.io/coreos/flannel                                   v0.9.1-amd64        2b736d06ca4c        10 weeks ago        51.31 MB
gcr.io/google_containers/k8s-dns-sidecar-amd64           1.14.7              db76ee297b85        3 months ago        42.03 MB
gcr.io/google_containers/k8s-dns-kube-dns-amd64          1.14.7              5d049a8c4eec        3 months ago        50.27 MB
gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64     1.14.7              5feec37454f4        3 months ago        40.95 MB
gcr.io/google_containers/pause-amd64                     3.0                 99e59f495ffa        21 months ago       746.9 kB

初始化Master

kubeadm init --kubernetes-version=v1.9.2 --pod-network-cidr=10.244.0.0/16
注意，这里的版本必须为1.9.2，否则安装不成功

大概显示如下内容：

[init] Using Kubernetes version: v1.9.2
[init] Using Authorization modes: [Node RBAC]
[preflight] Running pre-flight checks.
[WARNING FileExisting-crictl]: crictl not found in system path
[certificates] Using the existing ca certificate and key.
[certificates] Using the existing apiserver certificate and key.
[certificates] Using the existing apiserver-kubelet-client certificate and key.
[certificates] Using the existing sa key.
[certificates] Using the existing front-proxy-ca certificate and key.
[certificates] Using the existing front-proxy-client certificate and key.
[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"
[kubeconfig] Using existing up-to-date KubeConfig file: "admin.conf"
[kubeconfig] Using existing up-to-date KubeConfig file: "kubelet.conf"
[kubeconfig] Using existing up-to-date KubeConfig file: "controller-manager.conf"
[kubeconfig] Using existing up-to-date KubeConfig file: "scheduler.conf"
[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml"
[init] Waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests".
[init] This might take a minute or longer if the control plane images have to be pulled.
[apiclient] All control plane components are healthy after 27.003370 seconds
[uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[markmaster] Will mark node master1 as master by adding a label and a taint
[markmaster] Master master1 tainted and labelled with key/value: node-role.kubernetes.io/master=""
[bootstraptoken] Using token: d405c1.18b51150e22ffe72
[bootstraptoken] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstraptoken] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstraptoken] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstraptoken] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[addons] Applied essential addon: kube-dns
[addons] Applied essential addon: kube-proxy

Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ 
You can now join any number of machines by running the following on each node
as root:

kubeadm join --token d405c1.18b51150e22ffe72 192.168.128.26:6443 --discovery-token-ca-cert-hash sha256:936229f8381de8df72e8b0de8a349a0099f0d0fc0407ca17a5bffe2e6

上面显示的内容最好复制下来保存好，后面还有用。

次时使用

kubectl

命令会提示错误

The connection to the server localhost:8080 was refused - did you specify the right host or port?

解决办法：

对于非root用户
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

对于root用户
export KUBECONFIG=/etc/kubernetes/admin.conf
也可以直接放到~/.bash_profile
echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile

安装网络模块

这里使用flannel网络模块

kubectl create -f kube-flannel.ymal

收尾（可选）

默认情况下，为了保证master的安全，master是不会被调度到app的。你可以取消这个限制通过输入：

kubectl taint nodes --all node-role.kubernetes.io/master-

添加node节点（在node节点上面操作）

上面让保存内容的最后一段内容就是添加node节点的方法。

准备镜像

所需文件
flannel.tar
kube-proxy-amd64.tar
pause-amd64.tar

导入镜像
docker load < flannel.tar
docker load < kube-proxy-amd64.tar
docker load < pause-amd64.tar

[root@node27 ~]# docker images
REPOSITORY                                         TAG                 IMAGE ID            CREATED             SIZE
gcr.io/google_containers/kube-proxy-amd64          v1.9.2              e6754bb0a529        12 days ago         109.1 MB
gcr.io/google_containers/pause-amd64               3.0                 99e59f495ffa        21 months ago       746.9 kB
quay.io/coreos/flannel                             v0.9.1-amd64        2b736d06ca4c        10 weeks ago        51.31 MB

加入node

kubeadm join --token d405c1.18b51150e22ffe72 192.168.128.26:6443 --discovery-token-ca-cert-hash sha256:936229f8381de8df72e8b0de8a349a0099f0d0fc0407ca17a5bffe2e6

根据个人内容添加

查看节点情况

[root@master1 src]# kubectl get nodes
NAME      STATUS    ROLES     AGE       VERSION
master1   Ready     master    1d        v1.9.2
node27    Ready     <none>    1d        v1.9.2

如果STATUS都是

Ready

说明成功
如果有NoReady的情况，使用

kubectl get pods --all-namespaces

查看pod状态，使用

kubectl detele POD名 --all-namespaces

删除一次测试